{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 问题描述  \n",
    "运行课程给出的mnist代码  \n",
    "\n",
    "给出代码的运行log截图并提供心得体会文档描述对整个模型构建及训练过程的理解。  \n",
    "\n",
    "- 没有明显报错的正常的log输出 60分。  \n",
    "- 卷积的特性描述20分。  \n",
    "- 为什么卷积的效果要比全连接网络好，给出至少2点理由，每点10分。  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个实例中，我们使用 tensorflow 来实现一个简单的手写数字识别的网络，并用这个网络来做个简单的识别示例。\n",
    "\n",
    "首先导入一些用到的库。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:50.515650Z",
     "start_time": "2018-06-01T06:32:43.133728Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorboard\\compat\\tensorflow_stub\\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:39:48.304594Z",
     "start_time": "2018-06-01T04:55:17.674707Z"
    }
   },
   "source": [
    "先来看看数据长什么样子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.075054Z",
     "start_time": "2018-06-01T06:32:50.518290Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-2-87e0ba3ec99f>:1: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./data_hw10\\train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./data_hw10\\train-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:110: dense_to_one_hot (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.one_hot on tensors.\n",
      "Extracting ./data_hw10\\t10k-images-idx3-ubyte.gz\n",
      "Extracting ./data_hw10\\t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\contrib\\learn\\python\\learn\\datasets\\mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "(55000, 784)\n",
      "(55000, 10)\n",
      "(5000, 784)\n",
      "(5000, 10)\n",
      "(10000, 784)\n",
      "(10000, 10)\n"
     ]
    }
   ],
   "source": [
    "mnist = input_data.read_data_sets(\"./data_hw10\", one_hot=True)\n",
    "\n",
    "print(mnist.train.images.shape)\n",
    "print(mnist.train.labels.shape)\n",
    "\n",
    "print(mnist.validation.images.shape)\n",
    "print(mnist.validation.labels.shape)\n",
    "\n",
    "print(mnist.test.images.shape)\n",
    "print(mnist.test.labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:49:40.128071Z",
     "start_time": "2018-06-01T05:49:40.123888Z"
    }
   },
   "source": [
    "可以看到 images 里面有数量不等的图片，**每张图片是 28\\*28 (=784) 长度的一个一维向量**，所以用的时候需要先给它还原成28\\*28 的二维图片。**labels 中则是图片对应的数字的值**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.695746Z",
     "start_time": "2018-06-01T06:32:51.077167Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcoAAAHRCAYAAADqjfmEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3hUxfrA8XdSCCWhK0qvoSlFxV5QwILYe0O9KAo2rop69frzcr2WawXERlGx14uKDSvYUBBRBOlFOkjvIWV+fyTMnFmzw2azm00238/z+PjOzuzZ0ZPdOWfmzIzSWgsAACheSqIrAABAeUZDCQCABw0lAAAeNJQAAHjQUAIA4EFDCQCABw0lAAAeFbqhVEpppdR2pdR9EZbvp5TaVvS+1vGuH0qG85l8ojinQ4rKa6VUWrzrh5KprN9RVZEXHFBKaRFpo7VeUJQ+RkQ+DilWQ0TO1Vq/E+59KB+KOZ/1ReQ9EWknIqkiMltEbtVaf+d7H8oP37lRSl0uIi+IyNVa69GB15uLyGIRSdda55VNTRGJ4s6nUipVRIaIyN9EJEtEFojI8VrrTb73VSRJdcWmtf5GRDL3pJVS3UVkvIh8kqg6oVS2SeGXb76IaBE5Q0TGK6X25Qe0YlNK1RGRf4jIrETXBaU2RESOFJEjRGSpiHQUkV0JrVGMVeiu1whcLiJva623J7oiKDmt9S6t9VytdYGIKBHJF5E6IlI3sTVDDDwgIsNFZF2iK4LoFV3wDJLCXoE/dKGZWmsayopAKVVdRM4VkbGJrgtKRyk1QwqvUN8XkdFa67UJrhJKQSl1qIgcIiLPJLouKLUDRSRPRM5VSq1WSs1TSl2X6ErFWlJ1vYY4RwqvVicluiIoHa11J6VUVRE5S0SqJLo+iF7ReNZTInKD1rpAKZXoKqF0GotILRHJFpEWItJGRL5QSs3TWn+W0JrFUNLeUUpht+uLuiI/rQSjqBv2NRG5QynVOdH1QdQGisgMrfXkRFcEMbGz6N//1lrv1FrPEJHXRaR3AusUc0nZUCqlmohIdxF5McFVQeyli0jLRFcCUeshImcVddOtlsKHQB5VSo1IcL0QnRlF/07qG5Jk7Xq9TES+11ovTHRFED2l1OFS+Dc6RQqnh9woIg1E5MdE1gulcoWIVA2k/ycib4vImITUBqWitV6olPpGRO5SSt0ohRexF4jIRYmtWWwla0PZV0QeTnQlUGoZUvhkZEsRyRWR30TkVK31yoTWClELzq0TEVFK7RaRLVrrzQmqEkrvIim80FkvImtF5G6t9ReJrVJsVfSu1xwRmaaUujf4ota6ndb6L1eoSqkrlVKbit5XUEZ1ROSc86m1nqS17qy1ztJa19VaH6e1/npPYc5nhVDsd3QPrXX3kMUG7hGRX4vel9TdeRXUX86n1nqF1vpkrXWm1rql1vrZPXnJ8h2t0CvzAAAQbxX9jhIAgLiioQQAwIOGEgAAD+9Tr71SzmMAM0E+K3grLkuWcE4TJx7nlPOZOHxHk0+4c8odJQAAHjSUAAB40FACAOBBQwkAgAcNJQAAHjSUAAB40FACAOBBQwkAgEeybrMFAEi0lFQTzhvV1cmaddJTJj7t8gEmTvtiWvzrVULcUQIA4EFDCQCABw0lAAAejFECAGIirVkTJz3vgXomXtx9dEjpKiba1MrG9b+IS9VKhTtKAAA8aCgBAPCg6xXlWmqHbBPPGVDHyZt/9tMmLhB3C78UsdvKPbWphYnHPtbbKVdvzOSY1BOorNJaNjfx73fVd/L+2t1qXb3sKBM3+GadifNjV7WY4Y4SAAAPGkoAADzoekXCpTVp7KR/v2c/E792wrMm7ppR4JQrCFznFYibF7wG7F97gYkb3v6KU+q5CceYOG/5isgrDUdK1aombvq1cvKeavSdiVOVPS+zd+9wyt1yUl8T589dICi/VLp9SnX2v+qaeHHP8F2tLT//m5Nu2/93Exfsmh/D2sUed5QAAHjQUAIA4EFDCQCAR4Ubo1x185FOWgVmBVRdbxMb27nv23+yfei46vgpcakbIrfooSNMPOeSJ5284FSP4DSPgpDrug931DLxlG0tw37WwTWWmPiczC1O3soJM038QUd3+gn8guOSK163U3A+aPRKccVFRKT7zDNNrB51pxJkLPyl1HVKa97UxHlLlpb6eCje3BGdTby456iw5VpPvMLEbfr+7OSFPlVQnnFHCQCABw0lAAAeMel6XXud2x26qVOuicedOCIWH2G0rzI1bN4unWfiWinVnLy1l2038crh7n/2Y6t7mXj9+TVNnLdsedT1hN95veyUgdBVddypHvZa7slNrZxyn53U0cS+qR3fnXahiU9/5mknLzh15APp5q80HAv+ZTfindPtybDl2nxxlYnbDphr4oLtS5xy7l9BZOaNdM/Zeyc+YeILXrjZyWv6r++j+ASIiCwYerib7vNUIGW/oy0/c6eAZPefZeJozm95wR0lAAAeNJQAAHjQUAIA4BH1GOW8UXZsYE7vYU5ehkoPpqL9iBJzP9e1b2qNQOzmvdjsaxNf+kZ3E2+8uKlTjsfNS+nQA014bT07Vvjhjv2cYsGpHjO3NDRxzuB9nHILH7InMvve6k5e/my7JFZwOlD6s+7Jzw0MnKy43R1rb/RfxrSC9BGdnfTXFz8cSNn//0vz3KXpsvvZKTgFubtLXY/cngebeFwv9xmIjoGl1VA6u0+2v/Hjzhzq5KUqOzXImQJy5a9OOV1QHvcCKTnuKAEA8KChBADAI+qu16ePf9HEoV2e/13fxsRrd2dFdfz/TbPdK03HK0/JyCzvYa8JHur9qpMXXK3l5eYTTXzpq92dchsvsLtcMHUkClN+M2H/cwaYOHXVBqeYO9VjtYlW3O6uvjP7ODsV4JRRVzt5qbNtvL6fXQUoV09zygWnojR75Q+3HqH1r+TW3O52m+6bartbd2qb13fQLU656rk/xrQe2/5uv68HVnF/e7bpHBO3eGu9k5ccnYBlp95di03cqUpVJ6/X7NNMnH2PPR/5SdLVGoo7SgAAPGgoAQDwiLrrdegF55r4n11qOnn7vmtX38hf73arRSpbwq/AE43W4208+rneTt7q1+0qMdfVXmbiYDesiEjb/ra7sPnddL2Whp5qu2Ej7eKsus5d22Pk5uYmrrJmm5O3aIh9gvWFy2wXbXCRdRGRaTn2WpGNm/36Z38bNu+sueeZuPq48F2tKs3+5Khq1cKWC5V/oO12f7z982HLdZ92pYn3nTUn4uPjr25s9HnYvC1j7TBU7fmTy6I6CcUdJQAAHjSUAAB40FACAOAR9RilnmZXha/nPnFf7h/DLpjhjl08/3gfE1835OnQ4sarl9oViO68+9DYV6yS2nmG+/9yQzv7Zxkcl6z3mzsO2b/WEhN3+cCd2nFohn1fcArI1Bz32vCf/ey0klRxN5ZF5LLSd5l4e0he7omHmLju3UtM/EbLT0vwCZOKffW7kPO5z4NltxJYMtp8qd0l5NiqdiPto2ac7ZSr/dIPZVan8oA7SgAAPGgoAQDwiMnGzUBprLzAXfFl9nG2+zs4nSN0g+dgXrCrNTQvOAXksrevd8q1/Cr5H22PlZGjTnPS195qFyR/saWdf3Xt9yc75cY0s+czTUJ2JCilK8Zf66TbTK5cXYKxtun00I7zQjvedzcuyNSL4leJlJC/kXKw2g93lAAAeNBQAgDgUSm7Xpff6e47WNB1a0Tva5BquwjzTjjYyUv7clpocUQp+JRq8FrOfd2f13/ZCSZe9g+7SD9drdHb3jj0/79VTdl9IMc2+zIk13al3bLaPuH80YRuTqnc/e33a8GJoyKqU/2fS79hAqz962wp9vVq68Of+2jlnGLP/7qr7R6mBzRY5ZTbeq7928pbtVoSgTtKAAA8aCgBAPCgoQQAwKPCjVGmtWzupBf029/ET104MqJjdK/qrsCSqiK7Xmiclmnikc8Pc/IGNjs6omPgrxq+UcVJn9fITkM4oOZKE19b73unXKPAxsGh13wLH2hv4mpfTYlBLZH97J9Oun3udRG9r/VLdgehgrkLTdwizx0vXvTgERKJgSuOMnHdV91nA3RoYXil7dfASY9q+0oglSmllVq7lonPnDzfybsga7iJa6WE30mm44hLTNz4HMYoAQAod2goAQDwKLddr9vOO8zEfx5k2/N/n/26U+7CrI1RHL301wc9Px/kpLPlp1Ifs7Kq9p7bNZrzno2nBc5V/24DnHJb77WriHx54BtO3tH/siu0/DqtiYnZnDl6+fMWOukWdywMUzLkfREeP21HZFM9fhrdxcT1c5nuUyrp6U6yaVrpulvXDnSn3p15zUQT96+1MqR0ZBt375NV/GpBZYk7SgAAPGgoAQDwoKEEAMAjoWOUqmtHE9ce4S5b9FFzu+NApNM33t1u+9dn7mwcttwHD3V30qk59qHyy/9td0H4a5+6VWV1eti8yiStifv/OW/Z8rh9lp76m5PODGxScd4kd2eLca0/MvEBV9mpO03/xRhleaU8g5l5gZHOOvNyyqA2lYPe6i7fOXJzQxP7fv9S69cz8bK/tTXxb4OeimHtCm3eWdXE+8b86JHhjhIAAA8aSgAAPMq06/WPIe6jw3dfaB/pvyRrvZO3NM+uJj9ndx0T3/DaVU656qvsI+X7T1xn4vzf54WtRy0Jv7nr/H8EVqoI6XpYnLvNxM3f2yaV1c4z7A4QwWkYIiIf/GG70/c/c3aZ1WnzI02ddMEztjs9t83OMqsHonflRRPC5p23wHatp078OWw5lEz+ps1O+rXldkeP/rXsPK2jbv/RKdftXrtx8/mZX8S0TkP+7OCkG95op4fkxfSTIscdJQAAHjSUAAB4lGnXa+1ua510sLu1x++nO3m5T+xn4uDKLc0l/Eocka4AEqrguK4mPrP2mECOex2xoSCwePcU9wnMZBd8uvWCBz428U9bmjvlyrK7Nbjg8rkPut12KcKGvuVd6j77OOk2GQvCll33dHMTZ0liFsauDHY9bzeZyHk418QP7zc95p+Vq+0vdodJ/Uyc/Q93GC7vj2Ux/+yS4o4SAAAPGkoAADxoKAEA8CjTMcp6/dwpFa1vtrtBtBrsjj2mydIyqZOIyMZsu/LDUVXDXzv0n3mpietL+OknyeiPi+30i+Bj449P7+mUayWxH8swDj3QSZ7y/Ne2TrXd8a2CwDVg+rzIdilA2dp8fCsnfVp1O868Tbur71RdlyuIv5qv2uleP/7Hrj52bNXiSu9dvi4w8SE/XezkVXnbTvtr+ZL9/U/UFBAf7igBAPCgoQQAwKNMu17zVrmPdbcaXD4e817frfib/dm7dzjprKdqFVuuMmj0lV08Of2mVBPf1OVLp9yYG041cb1ZbvdZ2pfTij12aodsJ72yR30TZ55q/0a+OvAFp1xwCkhByDVf9sfX2HjI98V+LhLr8iHvh81bnOuez/TPi//bQWK0+/YyE6uZWSZuMXyWU07n267XfbfOiX/F4oQ7SgAAPGgoAQDwoKEEAMAjoRs3J8pJM7c46XG1nwyk7DJ1l8+63ClX5+Op8axW+RZYsu+oGWeb+MsD33CKXXvHEyYukAInb8jag4s99Om1XnPSXTPs+1IC13Khxwte57V9+zonp8PDdtmr8vi4OUTqpYbfgeeRVSeFvLIpvpWBV4enBzrp5g/YZUV1nv2GRbuMaHnHHSUAAB40lAAAeFTKrtdza85w0tVTMk08L9duElp9RO0yq1NFUvvq3SYe8r7bnXp/A/v/Nlc7WXLvvr+YuEBsZuhOH8GpHmvy7abLT613N/7+dMRRJm4zxl3Zie7Wim13QereCyGu7mvZxcRNxJ1ipUMLJznuKAEA8KChBADAo9J0va4daLvtGqS6T68uzrVP3110/2AT1/84/CbRlVnesuUm/vW0Jk5e6/8W/2SriMjs7qNNfOyM803854aaYd/TeqjtRNVT3c2y63k28UbFNqr5B0764Ef/buJWt/wQWhyIK+4oAQDwoKEEAMCDhhIAAI+kHaNUGRlO+pxr7S4XWwt2O3m9p9gNpJs+y7hXSeQtX+GkW12yIkxJkT5ixy9rysJAHF5lewy9Mrnr9UucdLu+j9k43f3+SoE7hQgoS9xRAgDgQUMJAIBH0na9SoHbaffS+ONN/PGv3Z28pm/yuDlQ1pr9nzvMcfP/HRG2bCumAiGBuKMEAMCDhhIAAA8aSgAAPJJ2jFLnulNAmt/FGAcAoOS4owQAwIOGEgAAD6U1a58AABAOd5QAAHjQUAIA4EFDCQCABw0lAAAeFbqhVEpppdR2pdR9EZbvp5TaVvS+1vGuH0omivPZs+h8Fiilesa7fig5vqPJJYrzOaSovFZKVdh5+xW6oSzSWWt9156EUmqkUmpu0Y/nFcGCWusxWuvMMq8hSiL0fJ6glPpZKbVFKbVIKdV/T57W+vOi87k0ITVFpPiOJpfQ89lFKTVNKbWj6N9d9uRpre8RkY4JqWUMJUNDGepXERkoIj8nuiIoHaVUuoiME5FnRaSWiFwgIo8ppTontGIoLb6jSUIpVUVE3hORl0WkjoiMFZH3il5PGknXUGqtn9RafyEiuxJdF5RaXRGpKSIv6UJTRWS2iHRIbLVQGnxHk0p3KVwKdajWOkdrPVxElIickNBaxVjSNZRIHlrrNSLymohcqZRKVUodISLNROTbxNYMQJGOIjJDuyvXzJAk6G4NqrCDq6g0XhOR0SIyrCg9QGu9LIH1AWBlisjmkNc2i0hWAuoSN9xRotxSSrUTkTdEpK+IVJHCq9TblFKnJrRiAPbYJoXDI0E1RWRrAuoSNzSUKM8OEJG5WusJWusCrfVcEflQRE5JcL0AFJolIp2UUirwWqei15NG0jWUSqkqSqmqUjignK6UqqqUSrr/zkpiuoi0KZoiopRSrUSkjxQ+NYkKiu9oUpkoIvkicqNSKkMpdX3R618mrkqxl4x/nJ+KyE4ROVJERhbFxya0RoiK1nqhiPxNRIaLyBYRmSQi74jImETWC6XGdzRJaK13i8iZUjg8skkKv69nFr2eNCp6Q5kjItOUUvfueUFr3V1rrUL+mSgiopS6Uim1qeh9BYmpMjyKO59vaq0P0Fpnaa0ba61v11oXiIgopXoUnc8GUnhVi/KH72hyKe58TtdaH6y1rqa1PkhrPX1PnlLqHinsAcoRkQq7pyP7UQIA4FHR7ygBAIgrGkoAADy8Cw70SjmPftkE+azgLbX3UiXHOU2ceJxTzmfi8B1NPuHOKXeUAAB40FACAOBBQwkAgAcNJQAAHjSUAAB40FACAOBBQwkAgAcNJQAAHjSUAAB40FACAOBBQwkAgAcNJQAAHjSUAAB4eHcPqcgWv97JSX971NMmvrjvDU5e6lc/l0mdAIS38NHDTXzTyR87eR9ddISJC2bMKbM6IQKH29/axTe5m2/MO26siVtPvMLJa3XxL3GtVixxRwkAgAcNJQAAHknb9aqX1nDS9Y6pZuINbTOcvH2+KpMqIcZyTu1m4g1Xb3Pypnd7JaJjXLv8GBN/+3FnJ6/ls4tMnLdqdTRVhEdao4ZOesQZz5u4V7WdTt7Yw3qbuN6M+NYLe7d60JEmvv/650x8YrXtTrncwBbUww593ckbLu2KPfaaG4500g1ftV3t+es3lLiuscAdJQAAHjSUAAB4JG3Xa43lKmzefhf84aTzn4l3bRAtlV7FxPMe6+rkfXja4yZune52pxdEePxnGn9j33P1105elwP7mrjxOXS9xtrCa5o56dDuViSWyrDfqY3nH+TkfX3royaurqpIaS3/h+1unXrdUCfvzesam3j40HOcvH2emVzqz44Ed5QAAHjQUAIA4EFDCQCAR9KOUfrszEt30qXvYUe8zH2ii4nnnfaUk5ciVU1cIFoi0X9Zdyc9usmksGWHd7GPsz9a7zgTJ+oR9WTT5Kjlia4CPBb9y45Lzuo7IiQ3sl/NZza1NPGzL53q5DWS702cU88+VZCuUp1yl2StMnG3Ox5z8i6Tm00cz/FK7igBAPCgoQQAwCNpu15rnroqbN7md9wVQfaRP8KURFkITgERcbtbZ/UJdvm4XTKr8neY+Nhxtzp5LcftNnHGfDu1I3/deqdc1zcuMfG0bi87eT/vbG5ivTs3TO1RErv6HGriYS2fCMlNFyRWcEpIjQ4bS/z+j3dkOel3bjvRxI0+/D60eIllh/xWvP6PR0x8UtdBttw1U0v9WUHcUQIA4EFDCQCABw0lAAAeSTVGmd/dPs48vuOTTt4vu+34VoNXZjp5kS53hvhYdd0hTnreacGxK3vexmxu6pT739W9TNzmux/CHj/P89k5OeHHxcavsBvSVtu62HMURGpnPXs+D6zCmGSiqTS3CVj4b/sb+vshoVNCiheccrX2HHeMMmNFZGOFzT+0zxR0anaFkzftiDEmDp060iLNThGrOSd+f0/cUQIA4EFDCQCAR3J1vWbYdj9TubtJ5Gq7ckvB1q1lVifs3YD+7znpFLE7vzywvoOJJ5+e7ZRTS36J6PipNWuaePlVBzh5t3X6n4mn73Y74audRHdronyX417DZy3zdaAjWjk93R15fr80su7Wm1YeZeI1p9ouz/z1K6OqR+pXP5u46Vdu3ri5+5v4/My1UR2/tLijBADAg4YSAACPpOp6XXIW7X5FlB9yvRZc4Pyj+7ubOGtJ+CdbJcV9Gi7/uM4m7jPiCxNfW9vt1wl2854698yQg64I/3mISrtrZ0VUbujyXk66yiexXWmlMltzo90keeCAdyN6T7CrVURk8XH2O1uwI/k3CaBlAQDAg4YSAAAPGkoAADySaowyaz+mfSSb6qt3772QuGOSIiIfvzwqovedtaC3iVPO2eHk5Ud0BJTEwAbBMWIVttzcj9s46cbyZ5xqlPxSOrd30g/eaFe66VFtR2hxI7jiTnAKiEh8xyVV145Ounn6z2FKiizIzTFxrUXxm0LEHSUAAB40lAAAeCRV1ysqpvk7G7gv1FpiwudeHG7iB9f0dIpN/KO1iT85dLi4qploc8EuE3f78O9OqXa32OkKBdu3R1plxFmzd92uVrrBo3fMS27Xpa+7NWjquweauNH60m+6HKm5A6o76UMzdJiSIhO225W7qr03JW514o4SAAAPGkoAADwqfNdrSlW7H9nRjcIvYj1q7XGB1LY41gglNfu6Du4L7/xowv1TbRfqsIbfOcVSGtruoIJAV2uo458YbOLsh9wuJPYijb/gSjBt04P//6s65VbkB7oE8+hsLY111xxh4gF1Hg3JtRtGrMrf6eTc/Iddnarp/9aYON5nI61FMxNPOvnxkNzw3+1vN7QOpNbFtlIB3FECAOBBQwkAgAcNJQAAHhV/jLJ2LRM/0fDjsOUmfWs37G0lnl0oUCZyTu1m4mUXuitqpHhWbAlKVYHrPO2ONvaYdbaJGz5Udo+2QyS1wb5OuuvFv5m4ZkrV0OJG93G3mrjNfL6jpbHVDvlJZkpG2HKPrD3efd8xwXG++I35hZp7nd2cOfhcQqiNgaleIiKrh7UycQ3GKAEASAwaSgAAPCp812te8wZ7LyQiTT/JjXNNECqlUzsnvd9IuxHy6CbPmji4UXNhunh3rO7mpP835RATP91rrJM3pu3LJu57vu3Sy3yTLr24q1/HSY5u8kmxxbaEdKNlLea6vax98vkhTrqFTC67D1d2iEWnesoF3Lr8FCdd4+0fw5SMLf4yAQDwoKEEAMCDhhIAAI8KP0a57q5dxb7ee87pTrrKxF9NHH4tepTWuv526awJdz/i5NVypgaEnwJyy6rDTfzxl3YMJftxd4nC7FV2t4BHjr/EyQtu3HzhPXba0AdvuuNniL38GlUiKvdbrrtLxH5DmcZT1vb/LnFLBW6+5DATzzn/yYje8/137nKXZTXVjztKAAA8aCgBAPCo8F2vTx/wSiBlnzFeuaWmU65h3vIyqlHlsvXCw510sLu1VsgqLLNz7RSdx1f3MvHcoR2dcrXe/cXELXfZx9Xd9XtcqZN+ddLt3rzOxL+eN9TE40683imX/ulPnqMiGlmProqo3IDpbnd5Y5kVpiTipdmdc5z0mvGxPX5a40Ymnn9dUyfvx0uDu5qEXz3ota12CmD28xudvLLqOOaOEgAADxpKAAA8KlzXa1pz9/Y9S9kn5VJVellXp9Jb18l9ejXY3Tpue10n7/nzTzVxwS+/mzgr5Mm1aDZTTqnmdvN2PGiJiTMCfxcFaZEtuI6SSWvS2MTZmUvDlrtkSU8TN7tqpZPHVs1l7+jaC5z0u23sUEr+/EURHSO1fRsTz7+8vpM39NznTXxite0h7wzf3Ro09rozTJw2a1pE74k17igBAPCgoQQAwIOGEgAAjwo3RrlrtJvOTrdjU/mBzXsz33Snh6BsBDddvv2r85287F+mxvSzUuvXM3H1ce7Y4xstPwqkGJeMt9W9m5j4/X3fd/KCG2xv3GVX40nZ7T7qr9Ltij46d3esq1iptBltp+gM6d3FybtnHzv96sqay5y81Pftb+hvOxpLJLrUmGTiS7IimxoU6v3tdsWsWz+/0Mlr94OdNhTN8wuxwB0lAAAeNJQAAHhUiK7X1OxWJr6l+fthy1202K72UvP1stnQs7KrP8NdYn5jwU4TT+091Mnr9uwgE7f/vz9MnL9mbdjjpzVqaOLtnRs5eYOGvWbiU6tvdvKCXTRPbrJ/P9W+mRO2HOIjOCTyUbvA93eeW67N2wNtfBMbbJdG3qIlJp4w/Ggnb9AQ+/82dPWsvjXt5uoSjGNgh3a705/cYLuEv/6b3ZQ9+6cpTrny8B3ljhIAAA8aSgAAPGgoAQDwqBBjlLsb1TJxj2o5YcvNe6OtiRtoNoEtC1mvu2NJx7YebOJfBzzh5M3r84yJZ51o9wIZNP+CsMd/pb3dHSZ0PCU4FSV0HCO4+fOcG+xmr2rrr4LYq7rBnoGFeTudvFZp1Yp9z86QMavqq7huj4e6z0120v83oIeJr91nopPXPj22y4AGnw94adgpTl79kcF6zYzp58Yaf5kAAHjQUAIA4FEhul59rl1+jIkbvjbXxOxEkBh159j/889saunkdahqN8/uXooyhf8AACAASURBVNV2m37W8R3PEauGzXlmczMTP/5hHyevzd3TTax20d0ab5lv2elY5+832Mn75R9Pmfg/69qZ+J2RJzjlGo1guKQsLOy2y8R3tL7IzbtiPxOfdLLd1PzR/d0hlo4v2g3QlefHttWr601c//fJ4QuWc9xRAgDgQUMJAICH0lqHzeyVcl74TMTVZwVvxWUl70Se0+Cm2/MfrB223AMHvWvi77e2NvH4CYc55VrcWbG6cuJxTvmOJk4yfkcru3DnlDtKAAA8aCgBAPCgoQQAwKPCTw9BxZG3ZKmJW1y4NGy5kRKcVmJXfGkhFWtMEkBy4I4SAAAPGkoAADxoKAEA8KChBADAg4YSAAAPGkoAADxoKAEA8KChBADAg4YSAAAP7+4hAABUdtxRAgDgQUMJAIAHDSUAAB4VuqFUSmml1Hal1H0Rlu+nlNpW9L7W8a4fSiaK89mz6HwWKKV6xrt+KLkozumQovJaKcXuRuVMZf3NrdANZZHOWuu79iSUUqcppWYWnZzvlVId9uRprcdorTMTU01EKPR8nqCU+lkptUUptUgp1X9Pntb686LzGX7PLpQHoee0i1JqmlJqR9G/u+zJ01rfIyIdE1JLRMqcT6VUfaXUd0qp9UqpTUqpyUqpo/YUTJbf3GRoKA2lVBsReUVErhWR2iIyXkTe58q0YlJKpYvIOBF5VkRqicgFIvKYUqpzQiuGqCmlqojIeyLysojUEZGxIvJe0euoeLaJyN9EZB8pPJ//FZHxyfabm1QNpYicJCLfaK2/1VrnSeFJayQixyW2WohSXRGpKSIv6UJTRWS2iHTwvw3lWHcp3DB+qNY6R2s9XESUiJyQ0FohKlrrXVrruVrrAik8j/lS2GDWTWzNYivZGkpV9E9o+oDEVAelobVeIyKviciVSqlUpdQRItJMRL5NbM1QCh1FZIZ2J3DPELpbKzSl1AwR2SUi74vIaK312gRXKaaSraH8TESOU0p1L+rKuVNEqohI9cRWC6Xwmoj8n4jkiMg3InKX1npZYquEUsgUkc0hr20WkawE1AUxorXuJIW9PxdLEl7IJlVDqbWeIyKXi8gIEVklIvVF5HcRWZ7IeiE6Sql2IvKGiPSVwguejiJym1Lq1IRWDKWxTQp/UINqisjWBNQFMVTUDfuaiNyRbM8RJFVDKSKitX5ba32A1rqeiNwjhV11UxNcLUTnABGZq7WeoLUu0FrPFZEPReSUBNcL0ZslIp2UUsEhkk5FryM5pItIy0RXIpaSrqFUSh1cNJ61jxQ+LTm+6E4TFc90EWlTNEVEKaVaiUgfEfk1wfVC9CZK4QMfNyqlMpRS1xe9/mXiqoRoKaUOV0odrZSqopSqppS6XUQaiMiPia5bLCVdQykiw0Rkk4jMLfr31YmtDqKltV4ohY+eDxeRLSIySUTeEZExiawXoqe13i0iZ0phd/omKTy/Zxa9joonQ0SeFJH1IrJCRHqLyKla65UJrVWMVejdQ5RSu6TwIY/hWuu7Iyh/pYg8LiJVRaSD1npRnKuIEojifPaQwoYzQ0R6a62/inMVUUJRnNN7RORmKTynNbTW+XGuIkqgsv7mVuiGEgCAeEvGrlcAAGKGhhIAAA8aSgAAPLwL1/ZKOY8BzAT5rOAttfdSJcc5TZx4nFPOZ+LwHU0+4c4pd5QAAHjQUAIA4EFDCQCABw0lAAAeNJQAAHjQUAIA4EFDCQCABw0lAAAeNJQAAHjQUAIA4EFDCQCABw0lAAAe3kXRgURIa9bExJsOa2TiVX12O+UGHDTJxIPqzHPyDvj2ShMXLKlh4tZDfnXKFezYEb4e++9n4rxVq/dWbSCp5PU42MTrO2Y4eTv3teu269bbTXx750+dcv1q2e/NJzvcYwwe2c/EDR/6vnSVjTPuKAEA8KChBADAg65XJNzKwUc66buues3EZ2WuDfu+lMB1XoEUOHkzjh5jE0fbsPOum5xyze4J3+WT8Ua+ifOODVsMIiLKbuO3dsARTtaAG941cf9aK6M6/MjNDU387umHm7hgyXKnnM51u+dRMpsvtf9vv3xwuIkzlNtUFEjxW2amiLudY6625XpUc4c5vr3xURMfmXqLiRs/UP66YbmjBADAg4YSAACPCt/1mtK5vYnn3lzNxJd1+dEpd0PdKSbu8ehgJ2+/oeXvVj/ZpXbINnGwq1UkfHfrn/k5TvqPvOomzpd0J++QKrYLLjXQLfjrVcOcct222K7Y/R91/w6OrrvQxBOkZrF1qtRSUk247K7DTPzbtSPCviVH2+7slXnu+awa6LXbN7W6k9evpu1i7TfxbRMP29jaKfdFnwNMnLdkadh6oHhbztxm4nRlz29oV+vSvJ0mvmv56WGP9+OclvZ4Ndxu8W+PetrER55pn0Zf9pj7dKzOcf9OEoE7SgAAPGgoAQDwoKEEAMCjQoxRqgzbZ726/8FO3o932DGnrQW2D/zw1291yn3dxY5lHHfpVCdv7tCYVBMlMOeOTBOHjkkGz+PxP11t4gbDqjrlUif+HPb4666xUxT6DPzaxHfW/8Upl+8Ohzi+3dAqkPozfMFKasXgSMcl80zc+VU7JtzytslOudT2bUw85x9ZTt7ME54xcXCqwk11Frgf9oENP+/ewsnKX7c+bB1RqPnVK0w88BM7J2rmhv2ccnUCs6zy5y2UcLJlQ9i8w575u4nnnWbHK7vccoNTrvH9iX+GhDtKAAA8aCgBAPAot12vKVVtN9ucoZ1MvOA0t4vniU22u+atISebuNWbId062bYbbUarLk6ePs0+l562wz6+nvbFtJJWGxH63zFPB1Lu9drAP+zj5g3P+j2q49d/1p7/L9fapXnuHPFLccWLNfcT+7fVmK5XUWnuz0WVoyLryjzgf7YrrU1Id2tQ/uz5tlxfN++Y/rav76HbR5q4e9Vcp1ywK/aLrAPdg9D1ulf5GzeaePooO3xRe6E7RSN/Xvhhj0ilbi/+Pq1j77lOevP9pf6oUuOOEgAADxpKAAA8aCgBAPAoN2OUKdXdJatWvNrMxAu62UfDH9vYxik34YbjTJz51Q9hjx98hLn6xi1O3qDJE008erV9JHrzF3upNKJ2YBW75Fzo8lhT59nH+rOl9ONKWTPt+OK3u9wpJvVm5YUWN7QKm1UppTZt7KSnHvxaseWe2NTSSbd7xo575YcWjlD9kXZsc9zVh5i4e8PwY54onXqjE/P/tk99d3P1V6RxmJJlhztKAAA8aCgBAPBIaNdrsLt1zqMHOHnB7tZHNrQ18dend3DKpS4u+WPKy65wu297VJtg4g372OO9WLuTUy5/0+YSfxaKd/zMc0z82QFvOnlju4828X3iTuWJVF4Pu4LTPvfabveWae45rH/LYhNvf889hip+b9pKa8kFDcPmbdN2+sDr95/s5NX6PfyQSDQWXdHcxN+Nd3cJOirDbuA9v79b35Z321VndF74LnfER84p3Zz0Fb0mFlvu3bVdQ15J/NQs7igBAPCgoQQAwCOhXa9/XtLZxAtOf9LJ+3CHXTT76zM6mjhv8ZJSf+7uWuH71Gbvst01dLXGT+Yg+6f39NtuV3j/WvNMPO+pQ03c4b+rnHJrTrRPw512/SQnr29tu1h+w7TgyufuKugvthxv4j693cWY86rR95par66Jb7/8zbDl3t5qn1Su9Upsu1pD5c+yK7dcPqG/k7fgdDtkM7uv+5ty6juB5X5+mhmfylVyqTXdDc7XXGR/u68Z5I5tBDfjXhLYCHr9w+5i9lXpegUAoHyjoQQAwIOGEgAAjzIdo0xr5D6ufdvgV028In+Hk/fAPQNNXHNR6cc80lo2N3GfU34MXxBlIrhTxEvDTnHyBtxj8+acERhnOsM9RkrgOq9ACtxMKX5H5ttXH+Gkx39tV3lp99tyJ++ah+zOJRPudsdeKgsV2MXnkqy1npKJUXNOyE/Y6cWXExGZe639b8m+Kk4VSlIpXdxpeSu71zbxlrZ2qs3VR7nPCgyu95XnqHbpq54f3Wzi7PFToqxl/HBHCQCABw0lAAAeZdr1WlDP7b46p4ZdLPnf6w5z8mq+WvLu1uDGsisGHerk3XH1Gya+MDPxjxtXdjvPsOfnmGumxvz4/f7oZeI/b25q4pQZC5xyrXfYvzPWaoneVxvbBVKbElYPlEza/vs56csn2YXQT6q+2sTp4naHpqvUUn/20bfa4bXsN2L/GxBL3FECAOBBQwkAgEe52Y/y9JrTnfQH/W8ycfqO8CukbDjVrujwwZFPmbhVmttV8O52+5RW6/evdfKCq3lM3dAskLPSX2mUyIYr7ROn59/yqYkH1ZkXUjKy67dg90+HJ91VdZrc930gZbsCQ5+N9UlRJSmdnBZd1TyicjNft09FNpDvPSVRnug67nDYWTU2BFJV4vrZzqYDBdHuVFo2uKMEAMCDhhIAAA8aSgAAPMp2eshvc5109pv28eB55z/l5E25x135PxKf7Kxn4jNH/83Ja/rQNBO3a7vFfWNgNY/5U+0YZUvGKEslrVkTJ333nWNNfEr1rSYOXVVnQ77dBPj0GfY8vnjAC0651ul29Z20XaWqarEKNNeRu5rtTnQVEE+r3Klyh0272MRd97UbXX/z5YFOuWprlBRnZwP3eZJ/n/O6ic/JXOfk9b5zook/ku4mzno9vrvPRINfAgAAPGgoAQDwKNvpIdq9LW/9d3uLfeic65y8gt4bpTib1mY56ebv2LjKJ3Z1hyYhj6gHP1nPmOPk/WfdASa+9CS7qO/3t8X38ehklNq2tYkfmPCyk9c23U7nWJpnu1d7vzzYKdf6qT9MXHeFnTrS5yX3b2TOCaNtuZNCuskfD6wcEuWj52NePdnEjZnygCSUv9H9nd3ndJsObhHQQiZLNF56wq649sTz1Z28Lw+0q6VNujqwefubIav+lIOpI9xRAgDgQUMJAIAHDSUAAB7lZgm7+s+G9IE/W3y5fWPwWan16jrprtXtWOm0HS1i8AmV1/x7Mk0cHJMUEfl8px1f/td9N5q4+fPuuQ+3i0fry9xlDs+ZdKqJJ3R8y8k7fKBdAnHfEdGNLza+n3FJn1WBzdZrLi3/e6/UWMAzB2Utb5XdgSTzZDfvlqlHm/ijdu+a+PCrr3fK/aVtSADuKAEA8KChBADAo9x0vZYl3cjtwD21+jYT3/SN3eEiW34qszolixcOfy5s3sM3XWbiuh+Wvjtl4SctbcLtrZGrBo438fsj6gliLyvFdq3n1LRxtTh/bmp7O5Xg0qsnRPy+ZmMXmbj8dxTHTmqdOk5a77arLRVs317W1TE++bqriR+/0A5znHXdV065b56tWmZ1Coc7SgAAPGgoAQDwqJRdryt61Q2bl7YuvQxrknxSA2sgpYRch2WszwktXirNX7BdaS/3dRdgP6raAhN/WD/bxPnr1se0Dskua1bgSdGT3LxMZRelP+ImuyrW7BfjW6dGL9hVmG6uMz9sufZj3ZWcWv45NUzJ5JPWpLGJO7y3wsn74D07vNR0SHyf7FYZ9m9k6eCDnbzber8bWrywTlXWhbzSuNhyZYk7SgAAPGgoAQDwoKEEAMCjUo5R5tTRey+EqLy8/kgTd234rZO35O82bvlABxMX/PJ7VJ+l8+yuApvz3Z0J2lex14Brz7JjlPVGRT4tZeuFh5u4PG4mWxaavL7EJm4OX+7A6navidmyX8zrsehBO672ZqPHAjkZTrlRm+1YdevHFzh5+XmVZ1LI5kMbmfjBBu87eXde9Z2JD67/dyev7eiQTe0jsOi82ibOreNuwn5vz7dNfH6mOx6aInbz5+C7nrr3XKdcLUn8d487SgAAPGgoAQDwqJRdr4ifTz8/yCb6ul2vM44eY+KV79mpIo+u7eGU+/ibrhKJcWcPNXHoAuzTc+w14D6v/Gpit2PI79x/fmriCa/XLME7k4cOrNwybGNrJ++mOrZr86KspSa+78XeTrm2j9jF0wtCNk0PZ9t5hznp6Zc+buJqgWkpwa5WEZH3z7Fd//l/hp86kuxqrNhp4uDG9CIi/6w/08Rzz37KyUs5O9gdGpzqpZxywTzn/RGWExFZG1hU/6j3bjFx9tvu5gflYaCMO0oAADxoKAEA8KChBADAgzFKEUlV9nqhzqwEViQJtB660MQ/XuAuB3hYRq6JG6fZPSYeDZlG8ugFbjqcFLHHLwgZffx4ayebt2OHRGPU7KNM3FR+i+oYFV3+ps0m/qKPO9YlH9gwOF45v8dop9hLh9rpIv993X30P+iSs7+0ca1HnbxqqnpocREReeLlM5x049lsti0iIj/MMOHXNx/hZJ34DzvW/L92bzh5wWUJQ8cbg9ypHXYU8ZWt7s5M52ba5QY7fjLQyWs2zh6jzYc/mrg8jEmG4o4SAAAPGkoAADzoehWRfG277erM3uYpib3JX7PWxA+efI6TN3fgPibu3+MLEw+qG93KPP2WHm/iqRPcbsGWY5YGUsslGk3Pq5zdreHkLVnqpF8dFthO5KZAWMddEeeyrNU2vnpEhJ/mdrW+sKWhid859zgTN579o8Av7Ytp7gv2qyenn3aTk7XyIrup85Rj7NSRc+de6JRb94Hd0UMFRj0avuJO/xnb2XaNZ3/5U8R1Lm+4owQAwIOGEgAAD7pexX3qFbGTP2+hk249yKa/lBqBuFuUn2AXcG4q7tOOlWf568QJLjD/6Qv1Tfx58y5OuTnX2ychjz7UdrN/O6WDhNNu5EYnXTBvsYl17tySVxbFqjp+ipNuOd7GF4pd5ShN3G73/ULSe+SHpNO+3FCq+pUXtBAAAHjQUAIA4EFDCQCAB2OUIrIw104JSd1kV3EJ7W8HUDyda6cV5M9f5OS1ucmm1wRf92zIy3cP5Ql3lAAAeNBQAgDgUSm7Xpv/c7KTHvjPowMpd0oDAKBy444SAAAPGkoAADxoKAEA8KChBADAg4YSAAAPGkoAADyU1jrRdQAAoNzijhIAAA8aSgAAPGgoAQDwoKEEAMCjQjeUSimtlNqulLovwvL9lFLbit7XOt71Q8lwPpMP5zS5RHE+hxSV10qpCru2eIV+6lUppUWkjdZ6QeC1kSJynIi0EZG/aa1fiOR9SLzQ86KUyhaRh0XkSBFJFZGpInKj1nqu730oP4o5p8eIyMchxWqIyLla63fCvQ/lQ5jf3C4iMkZE2ovIbBHpp7X+JZDfXEQWi0i61jqvTCscIxX6jjKMX0VkoIj8nOiKoNRqi8j7ItJWRBqIyBQReS+hNUKpaK2/0Vpn7vlHRPqIyDYR+STBVUMUlFJVpPA7+bKI1BGRsSLyXtHrSSPpGkqt9ZNa6y9EZFei64LS0VpP0VqP0Vpv0FrnisjjItJWKVUv0XVDzFwuIm9rrbcnuiKISncp3K5xqNY6R2s9XESUiJyQ0FrFWNI1lEhqx4rIaq31+kRXBKWnlKouIudK4V0IKqaOIjJDu2N4M4peTxo0lKgQlFKNReRJEbk50XVBzJwjIutEZFKiK4KoZYrI5pDXNotIVgLqEjc0lCj3lFL7iMinIvKU1vq1RNcHMXO5iLyoK/IThdgmIjVDXqspIlsTUJe4oaFEuaaUqiOFjeT7WuuIHklH+aeUaiKF41svJrgqKJ1ZItJJKaUCr3Uqej1pJF1DqZSqopSqKoUDyulKqapKqaT776wMlFI1RWSCiHyntb4j0fVBTF0mIt9rrRcmuiIolYkiki8iNyqlMpRS1xe9/mXiqhR7ydiAfCoiO6Vw7t3IovjYhNYI0TpLRLqJyJVFk9D3/NM00RVDqfUVHuKp8LTWu0XkTCk8n5tE5G8icmbR60mjojeUOSIyTSl1754XtNbdtdYq5J+JIiJKqSuVUpuK3leQmCrDwzmfWuuxReevRnDundZ6qQjns4L4y3dURERr3U5rPSa0MOe03CvuN3e61vpgrXU1rfVBWuvpe/KUUvdI4dz2HBGpsGPRFXplHgAA4q2i31ECABBXNJQAAHh4V3PvlXIe/bIJ8lnBW2rvpUqOc5o48TinnM/E4TuafMKdU+4oAQDwoKEEAMCDhhIAAA8aSgAAPGgoAQDwoKEEAMCDhhIAAA8aSgAAPGgoAQDwoKEEAMCDhhIAAA8aSgAAPGgoAQDw8O4eUtHMH3uQief2HOXknXD9QBNXH/djmdUJACqL1I5tnfSSs+uZ+JDeM528F5t9beJcnR/R8XtcN8BJV3t3SkmrGBXuKAEA8KChBADAI6m6XkXbPTcLpMDJWtHDxm3GlVWFEJTWopmJl53VyMRbs/Occm2zV5h4fNv3TZz9wbVOucYT7HVezemrnTy9bYeJ8//808Qqzf2TX3njoSbOq+bWt+kj0+zxcnIEwF9tufhwE596x0Qnb1y938K+L1fb72/o73U4Tw8d5qQHz+1r4vzZ8yM6RjS4owQAwIOGEgAAj+TqevVo1X6liVVGhpNHt1p8rB50pJP+afATJo60qyVYal6fZ9y8PuGP8cbW/U383N/PMvHKY9w/+d8ud7tygk6beLWJ1Xe/7K2qQNJKqVrVSS/8V1cTz7pshIkj/V5HKzu9ipOefVMdm3dtaOnY4Y4SAAAPGkoAADxoKAEA8Kg0Y5QftXvXxGdk9nLy8hmjjJnU1i1MPPamx0NyS/7nNm7bviY+J3NdxO+7IGuVjUc/ZeKUkGvD4IjK9Bw3L3XzrmLLVSZrbrTjzFsO2eUpGV/pGXYK0cyjnw9brk+jg8uiOpWDstPtgmOSIiK/XTY8kCr9/VaHN28Im/f7+U+EzXvg+LdM/PyhfWzGlPDTUqLBHSUAAB40lAAAeFSarleUjZW97bSM9lXCX4ed8NsFJq5xb82w5dJXbTLxmIa1nbycevZR8YEPveXknZW5du+VFZGZu7WJB98y0MmrPpPF87cfblc4mn3cqLDlgl3a0U4RiPQYwZyXtzSJ6rNQvIJjbBfrov729d9PGF5M6b96e9t+Tvqf39qpWU3ed38Pqr1nFzRvLT+YWHXt6B70/PCfF/yeD29Zw8RZMV4rnTtKAAA8aCgBAPCgoQQAwIMxSsTU0ZdNC5u3Kn+nidf81sDEqaeEP16Dn+w45JpDUt3P6mkfAY90TDLUB1u6mJgNvf+qzcDFJj476ywnb/EVTU2cU8eOHCotUSmov9vEs3s+G7Zcu4/sWHL72xaE5G6M7sMrq8AUEJHQccmRER3itLmnm7jg7n2cvOzvfoq+buUId5QAAHjQUAIA4EHXK2Lqw586m/ih075x8pqmZZp49sUjJCJX2jBduV2vuTo/kHKv+dYFunmPeetWE0887xGn3J31bfdt9/Ovc/Iy3/xBKrv8TZttIhiLSJN7l8f0s7adbzcAlp5u3oJcuzJP+4c32PptpKu1pII7gYSuuBPpNJAfc9JNrE+wG60rWVFc8QqPO0oAADxoKAEA8KDrFTGVPcAuiXFQvX5O3m9HvWDiaFZvyQ15mvL97XbT1mGLezh5KcPqm7jVR7YL9ZgaNzvl5pz2pIlX9sp38rLfLHEVUQqr+uwOmzdkuV3wOn/ewrKoTtLS7VuZ2F3cPLz2X1zjpFuNtN/fFEn+Tc25owQAwIOGEgAADxpKAAA8GKNE3LQc4o45de94XZiS0an902oTV1u0OCQ3NL13B2Yvc9Js51225vcYbeKCkGv4aVPamLi1rC+zOiWjFT1qmTh0I/OgcdvrmrjNiFw3M8YbI/sE6/jXKWI21u4iQzGuAwAACIuGEgAAD7peETf5s+Y66cxZsT1+3t6L/EXb7PArh/w2z90EOFtWhymJeCgQHYjd6UPRLrQOkbQmjZ30yRdPNrFvmtbtX9rN1bOnxHgnZI/ld7vpYB1Dp4hdvsQu4VTnw99N7E70Kj3uKAEA8KChBADAI7m6XgP9M6FPc4U+LYXKI7fnwSae0NbdY29yYHHntk/tcPLo7YuvnWccGvJK+L1M8+vapy4XvWr3ED242VKn3KD9P7PvEfcxyKufu97ETf7zfUmqWqFtOMbtev1Pg3Fhy/aaeb6J2982x8Sx7soMteSNTiZ+rssLEb9v4TPtTFx7y2RPydLhjhIAAA8aSgAAPGgoAQDwSK4xysDSDKGPPQcfK559fysnL/uaDYLkkpKVZeL7R9pxydCx6q+32TEOPT3G81eSTGqDfZ301iNbmHhnXXvNnXL2uoiON7bj0JBXMsKWnXPiMxEds98fvUw87ZMOTl7zx+wuFyXfu6biWn/6jr0XKrJseT0TZ28p+epW0bqt06cmPiQj/Ihov6XHO+l6nywwcTzHUbmjBADAg4YSAACP5Op6jVSVytTxUjmk1qvrpLe9ahd+7poRfmWP5yYdZ+I28mN8KleB5Z54iImz7l7i5I1rOcLEwelYkW/Knb73IkWCXap/3tw0fMEfZpiwqbhTQCrrt/7OLp84ad9C6Nn9fop3dYwtH9shsL41g1ODwtfv9+c6Oul6f8ZvSkgQd5QAAHjQUAIA4EFDCQCAR+Uco0TSWdavnZP+6YBhxZb7z7pOTrr942tMHM1uJMnuj1PsT8SElhOcvFe2NjLxpvzqJn5vZWen3NqvGklxhvd71kn3qGYf8O/280VOXt0+8wKpTf5Kw5Gv3fuhyMeQSy+1tn1WYMEzzZy8WZ2ej6hOHd68wcStR5XNmGQo7igBAPCgoQQAwIOuV1QYwdV2RETWvNLQxO90fjikdBUTDd9ou2UnPHSMU6rWoh9iV8EkVHu2Xe0q+6Nrnbz2g213aP6mzSauIn845RqHpPf49WK3K+7YqvOjrifKh+BOPSIiDe6153Rc0zEhpYu/T/t8p/s9bzvKrpwW711MwuGOEgAADxpKAAA86HpFTK0edKSJq/R0F8d+tMObJi7QkV2j3bfkVBMPafGukxdccSfY1Rrqqwvs6jK1ZtHVWhL1R04OxG5ePLvB0l+uu/dCiLn1Vx1h4nqjI3vCdN7ztru1WaP1Tt6opl+UuA43E5sXwwAAA2hJREFUfHy5k27ze+JXzOKOEgAADxpKAAA8aCgBAPBgjBKlsvWCw530T4OfCFs2uGlyrs6N6PgftbPjkqGbLgd3AtlcsMvJ6/HIYBPvN8vdRQKJE9z8uWF68dNGRETSdlXWvT5i76EZJzrpvkc/H6akSId+dvPyn/a3zxv0v/Ajp9x1tReaOF3ZDbFzdejIdfh7seD3OXvs9SZu84/ErL7jwx0lAAAeNJQAAHhU+K5XlWb/EzJq7E5gTSqnVb3cpcR9ixsHu0qjWZg5dNPl4DH+b3UPJ6/Rp3+aOFGreeCvth7ZwsRnZX4Ykst1ezw0HulukD25m+3yPCzDHQJxpnNcG35qR/DbG+n3OrhClojIqPG2S7jlv342ccjXvFzgLxMAAA8aSgAAPGgoAQDwqPBjlCktmpr4lyOfC1suOH2g4UcV/j87oVLr2eXFLjp4SgJrYj3e8Bsn/dX4TBM/cXR3E+etXiMoH1JCrtOD39G0bYwsx0raF9Oc9K1DBpj4m/uHx/SzluflOOmH1vQy8bIrmjh5LX6300DK47hkEHeUAAB40FACAOBR8fsgN2wy4YEv3mjiQ46d4xRb/nAbE2e+m/jV6Cuy3A52w9179p0Q8+Of/Pu5Jl4zqZHNUG65Oy6xu5FckLXKyTu+2jYTP5ERfmcRJE7oVIIXNx9o4vTPp4UWR4zU/8SuqtO1yU1O3vQBw0p17LOG3eak938suCrWPKmouKMEAMCDhhIAAI8K3/Wav36DiVsEFtNdH1KumpSPpzOTQfoq29199PRLnLxvu74S9n2r8neauNdLdtHy1iOXO+UyVtpu1Ca54RfOfv2ZriZ+o/oRTt6WgxuaOGtLxe3yqUxGzT7KxE3ltwTWJLnlr1lr4ib/Wevknf6fbqU69v6SnBsQcEcJAIAHDSUAAB40lAAAeFT4MUqUvfwFi01ct4+bd7pENsbRXOx4cp6nnLcef/4ZNq/6H8tsuSiPj9hb0TN8XuZHmeEzgQTijhIAAA8aSgAAPOh6BVBmUnbZ5ZUeXX+Ak1f3+cmhxYFygTtKAAA8aCgBAPCgoQQAwIMxSgBlptUtP5h4klRLYE2AyHFHCQCABw0lAAAeSmud6DoAAFBucUcJAIAHDSUAAB40lAAAeNBQAgDgQUMJAIAHDSUAAB7/DxK8xwtogAslAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 8))\n",
    "\n",
    "for idx in range(16):\n",
    "    plt.subplot(4, 4, idx + 1)\n",
    "    plt.axis('off')\n",
    "    plt.title('[{}]'.format(np.argmax(mnist.train.labels[idx])))\n",
    "    plt.imshow(mnist.train.images[idx].reshape((28, 28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，定义用于训练的网络，首先定义网络的输入。  \n",
    "\n",
    "这里我们直接使用上面的数据作为输入，所以定义**两个 placeholder 分别用于图像和 lable 数据**，另外，定义**一个 bool 类型的变量用于标识当前网络是否正在训练**。  \n",
    "\n",
    "为了让网络更高效的运行，多个数据会被组织成一个 batch 送入网络，两个 placeholder 的第一个维度就是 batchsize，因为我们这里还没有确定 batchsize，所以第一个维度留空。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.706044Z",
     "start_time": "2018-06-01T06:32:51.698913Z"
    }
   },
   "outputs": [],
   "source": [
    "x = tf.placeholder(\"float\", [None, 784], name='x')\n",
    "y = tf.placeholder(\"float\", [None, 10], name='y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为我们输入的是图片展开后的一维向量，所以**第一步就需要先把一维向量还原为二维的图片**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.719298Z",
     "start_time": "2018-06-01T06:32:51.707730Z"
    }
   },
   "outputs": [],
   "source": [
    "x_image = tf.reshape(x, [-1, 28, 28, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:07:36.530623Z",
     "start_time": "2018-06-01T06:07:36.522665Z"
    }
   },
   "source": [
    "接下来，我们定义第一个卷积层，**使用 6 个 5\\*5 的卷积核**对输入数据进行卷积，padding 方式选择 valid(不填充)，所以**输出数据的宽高变为 24\\*24**, 但是**深度已经从原来的 1 变成了 6**。  \n",
    "本卷积层的**激活函数为 relu**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.764292Z",
     "start_time": "2018-06-01T06:32:51.721295Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Entity <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3FB6CA48>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3FB6CA48>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3FB6CA48>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3FB6CA48>>: AssertionError: Bad argument number for Name: 3, expecting 4\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('conv1'):\n",
    "    C1 = tf.contrib.slim.conv2d(\n",
    "        x_image, 6, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来进行 **stride 为 2 的最大池化(max pooling)**，池化后，输出深度不变，但是长宽减半，所以**输出变成了 12\\*12，深度 6**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.774112Z",
     "start_time": "2018-06-01T06:32:51.766784Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Entity <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA3AA08>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA3AA08>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA3AA08>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA3AA08>>: AssertionError: Bad argument number for Name: 3, expecting 4\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('pool1'):\n",
    "    S2 = tf.contrib.slim.max_pool2d(C1, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:10:16.678485Z",
     "start_time": "2018-06-01T06:10:16.671472Z"
    }
   },
   "source": [
    "接下来，我们定义第二个卷积层，**使用 16 个 5\\*5 的卷积核**对输入数据进行卷积，padding 方式还是选择 valid，**输出 8\\*8, 深度为 16**，本卷积层的**激活函数为 relu**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.805912Z",
     "start_time": "2018-06-01T06:32:51.776959Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Entity <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3F9D2DC8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3F9D2DC8>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3F9D2DC8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Conv.call of <tensorflow.python.layers.convolutional.Conv2D object at 0x0000022E3F9D2DC8>>: AssertionError: Bad argument number for Name: 3, expecting 4\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('conv2'):\n",
    "    C3 = tf.contrib.slim.conv2d(\n",
    "        S2, 16, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再进行一次 **stride 为 2 的最大池化(max pooling)**，**输出为 4\\*4, 深度16**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.814748Z",
     "start_time": "2018-06-01T06:32:51.807560Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:Entity <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA97B88>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA97B88>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA97B88>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Pooling2D.call of <tensorflow.python.layers.pooling.MaxPooling2D object at 0x0000022E3FA97B88>>: AssertionError: Bad argument number for Name: 3, expecting 4\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('pool2'):\n",
    "    S4 = tf.contrib.slim.max_pool2d(C3, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "池化后的数据是 3 维的，这里做一个**拉平的操作，将 3 维数据展开到 1 维**，然后送入两层全连接，**全连接隐层中神经元个数分别为 120，84**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.856740Z",
     "start_time": "2018-06-01T06:32:51.817287Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\contrib\\layers\\python\\layers\\layers.py:1634: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use keras.layers.flatten instead.\n",
      "WARNING:tensorflow:Entity <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x0000022E3FACD688>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x0000022E3FACD688>>: AttributeError: module 'gast' has no attribute 'Num'\n",
      "WARNING: Entity <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x0000022E3FACD688>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Flatten.call of <tensorflow.python.layers.core.Flatten object at 0x0000022E3FACD688>>: AttributeError: module 'gast' has no attribute 'Num'\n",
      "WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FAD3B08>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FAD3B08>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FAD3B08>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FAD3B08>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3F9083C8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3F9083C8>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3F9083C8>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3F9083C8>>: AssertionError: Bad argument number for Name: 3, expecting 4\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('fc1'):\n",
    "    S4_flat = tf.contrib.slim.flatten(S4)\n",
    "    C5 = tf.contrib.slim.fully_connected(\n",
    "        S4_flat, 120, activation_fn=tf.nn.relu)\n",
    "\n",
    "with tf.name_scope('fc2'):\n",
    "    F6 = tf.contrib.slim.fully_connected(C5, 84, activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:17:39.880958Z",
     "start_time": "2018-06-01T06:17:39.869797Z"
    }
   },
   "source": [
    "对特征添加一个 **0.6 的 dropout**，以 40% 的概率丢弃特征中的某些数据，这样可以提高网络的推广能力，**减少过拟合**的可能性。  \n",
    "    \n",
    "需要注意的是，*dropout 仅在训练的时候使用**，验证的时候，需要关闭dropout，所以验证时候的 keep_prob 是 1.0。\n",
    "  \n",
    "dropout 的输出最终送入一个隐层有 10 个节点的全连接层，这个全连接层即为最后的分类器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.913419Z",
     "start_time": "2018-06-01T06:32:51.860766Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-11-413a88f1eb86>:3: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n",
      "WARNING:tensorflow:Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FA9CA08>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FA9CA08>>: AssertionError: Bad argument number for Name: 3, expecting 4\n",
      "WARNING: Entity <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FA9CA08>> could not be transformed and will be executed as-is. Please report this to the AutgoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: converting <bound method Dense.call of <tensorflow.python.layers.core.Dense object at 0x0000022E3FA9CA08>>: AssertionError: Bad argument number for Name: 3, expecting 4\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('dropout'):\n",
    "    keep_prob = tf.placeholder(name='keep_prob', dtype=tf.float32)\n",
    "    F6_drop = tf.nn.dropout(F6, keep_prob)\n",
    "\n",
    "with tf.name_scope('fc3'):\n",
    "    logits = tf.contrib.slim.fully_connected(F6_drop, 10, activation_fn=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来定义 loss 和用于优化网络的优化器。loss 计算使用了 sparse_softmax_cross_entropy_with_logits, 这样做的好处是labels 可以不用手动做 one_hot 省了一些麻烦。这里**使用了 sgd 优化器，学习率为0.3**。\n",
    "\n",
    "> 试试看，增大减小学习率，换个优化器再进行训练会发生什么。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.084738Z",
     "start_time": "2018-06-01T06:32:51.915376Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-12-034b5f273ff3>:2: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See `tf.nn.softmax_cross_entropy_with_logits_v2`.\n",
      "\n",
      "Conv/weights:0\n",
      "INFO:tensorflow:Summary name Conv/weights:0 is illegal; using Conv/weights_0 instead.\n",
      "Conv/biases:0\n",
      "INFO:tensorflow:Summary name Conv/biases:0 is illegal; using Conv/biases_0 instead.\n",
      "Conv_1/weights:0\n",
      "INFO:tensorflow:Summary name Conv_1/weights:0 is illegal; using Conv_1/weights_0 instead.\n",
      "Conv_1/biases:0\n",
      "INFO:tensorflow:Summary name Conv_1/biases:0 is illegal; using Conv_1/biases_0 instead.\n",
      "fully_connected/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected/weights:0 is illegal; using fully_connected/weights_0 instead.\n",
      "fully_connected/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected/biases:0 is illegal; using fully_connected/biases_0 instead.\n",
      "fully_connected_1/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/weights:0 is illegal; using fully_connected_1/weights_0 instead.\n",
      "fully_connected_1/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/biases:0 is illegal; using fully_connected_1/biases_0 instead.\n",
      "fully_connected_2/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/weights:0 is illegal; using fully_connected_2/weights_0 instead.\n",
      "fully_connected_2/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/biases:0 is illegal; using fully_connected_2/biases_0 instead.\n"
     ]
    }
   ],
   "source": [
    "cross_entropy_loss = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "l2_loss = tf.add_n([\n",
    "    tf.nn.l2_loss(w)\n",
    "    for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)\n",
    "])\n",
    "\n",
    "for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES):\n",
    "    print(w.name)\n",
    "    tf.summary.histogram(w.name, w)\n",
    "    \n",
    "total_loss = cross_entropy_loss + 7e-5 * l2_loss\n",
    "tf.summary.scalar('cross_entropy_loss', cross_entropy_loss)\n",
    "tf.summary.scalar('l2_loss', l2_loss)\n",
    "tf.summary.scalar('total_loss', total_loss)\n",
    "\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.3).minimize(total_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:25:56.449132Z",
     "start_time": "2018-06-01T06:25:56.438340Z"
    }
   },
   "source": [
    "需要注意的是，上面的网络，最后**输出的是未经 softmax 的原始 logits**，而不是概率分布，要想看到概率分布，还需要做一下softmax。\n",
    "\n",
    "将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:39:50.010829Z",
     "start_time": "2018-06-01T06:39:49.997501Z"
    }
   },
   "outputs": [],
   "source": [
    "pred = tf.nn.softmax(logits)\n",
    "correct_pred = tf.equal(tf.argmax(y, 1), tf.argmax(logits, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "saver 用于保存或恢复训练的模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.127795Z",
     "start_time": "2018-06-01T06:32:52.103115Z"
    }
   },
   "outputs": [],
   "source": [
    "batch_size = 100\n",
    "trainig_step = 1100\n",
    "\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上定义的所有操作，均为**计算图，也就是仅仅是定义了网络的结构**，实际需要运行的话，还**需要创建一个 session**，并将数据填入网络中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:35:22.270272Z",
     "start_time": "2018-06-01T06:33:18.829198Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "After 100 training steps, the loss is 0.192577, and the validation accuracy is 0.9398\n",
      "After 200 training steps, the loss is 0.241785, and the validation accuracy is 0.9478\n",
      "After 300 training steps, the loss is 0.21664, and the validation accuracy is 0.9686\n",
      "After 400 training steps, the loss is 0.0794006, and the validation accuracy is 0.971\n",
      "After 500 training steps, the loss is 0.0687732, and the validation accuracy is 0.9744\n",
      "After 600 training steps, the loss is 0.199552, and the validation accuracy is 0.9762\n",
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:960: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "After 700 training steps, the loss is 0.0917674, and the validation accuracy is 0.9662\n",
      "After 800 training steps, the loss is 0.0862564, and the validation accuracy is 0.983\n",
      "After 900 training steps, the loss is 0.114775, and the validation accuracy is 0.9796\n",
      "After 1000 training steps, the loss is 0.148875, and the validation accuracy is 0.9804\n",
      "The training is finished!\n",
      "The test accuarcy is:  0.9816\n"
     ]
    }
   ],
   "source": [
    "merged = tf.summary.merge_all()\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"./logs_hw10/\", sess.graph)\n",
    "\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    #定义验证集与测试集\n",
    "    validate_data = {\n",
    "        x: mnist.validation.images,\n",
    "        y: mnist.validation.labels,\n",
    "        keep_prob: 1.0\n",
    "    }\n",
    "    test_data = {\n",
    "        x: mnist.test.images, \n",
    "        y: mnist.test.labels, \n",
    "        keep_prob: 1.0\n",
    "    }\n",
    "\n",
    "    for i in range(trainig_step):\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        _, loss, rs = sess.run(\n",
    "            [optimizer, cross_entropy_loss, merged],\n",
    "            feed_dict={\n",
    "                x: xs,\n",
    "                y: ys,\n",
    "                keep_prob: 0.6\n",
    "            })\n",
    "        writer.add_summary(rs, i)\n",
    "\n",
    "        #每100次训练打印一次损失值与验证准确率\n",
    "        if i > 0 and i % 100 == 0:\n",
    "            validate_accuracy = sess.run(accuracy, feed_dict=validate_data)\n",
    "            print(\n",
    "                \"After %d training steps, the loss is %g, and the validation accuracy is %g\"\n",
    "                % (i, loss, validate_accuracy))\n",
    "            saver.save(sess, './model_hw10/model.ckpt', global_step=i)\n",
    "\n",
    "    print(\"The training is finished!\")\n",
    "    #最终的测试准确率\n",
    "    acc = sess.run(accuracy, feed_dict=test_data)\n",
    "    print(\"The test accuarcy is: \", acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:58:16.766415Z",
     "start_time": "2018-06-01T06:58:15.918700Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\ilove\\Anaconda3\\envs\\tfl1.14\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to check for files with this prefix.\n",
      "INFO:tensorflow:Restoring parameters from ./model_hw10\\model.ckpt-1000\n",
      "1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcoAAAHRCAYAAADqjfmEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzdd5gUxdYG8Pfssiw5R0krOSkqGDGAOWfFKwaM12vAcNXPgIo5XHOOiFmuYvaKAUVBFAQElKgCkkGQHJZl93x/VG131zDdzE7YwL6/59lnT0/VVHdPTXdNV3UQVQURERHFl1XWC0BERFSesaEkIiKKwIaSiIgoAhtKIiKiCGwoiYiIIrChJCIiisCGkoiIKEKpNJQiMlhECkRkvYjUTPA9f4jIFhF5PSKPisgGEbk7fUtb+kTkaxHZLCJjynpZEsH6jMb69PKwPsuIiAy19TMvwfwdbf0XisiFIXn6iEiRzXdkWhe4FIlIrl2HAhG5K5H3JNxQ2oKDf4Ui8kQJlm+YqtZS1Q22vM9iytsiIr8UZ1bVdgDuSaDcHqp6c2A5nxeRWbZCB8RZj6tFZKmIrBGRISKSG0jLE5FvRGSjiMwUkUPDZmo/7CEistaWd00grZWI/Cgif4vIQzHvGyEivYKvqerBAC5JYF3Twi77SyLyp4isE5GfReSoEhYTW5/Xicivtry5InJdMDPrM7NE5HIRmSAi+SIyNIkiYuuzr/3s1sTb2bI+M09EGojI+2J+bPwpImeWsIgHVDUvUF7oZ6Kqs1W1FoDR2ylzsf2ejLBlNheRj0RksZgfRnnBzFHztOmH2LrcaOu2TdiMo+rfljNXRJaISL/A6/VEZJKI1A6sa75d1ze2s66ehBtK++HUsjNoCmATgHcSfX+c8o6KKXNsKuUFTAFwKYBJsQkicgSAGwAcAiAPQFsAtweyvAXgZwANAdwM4F0RaRwyn8EAOgBoA6AvgOvF/5V1I4BXAOwM4MTiDc9W4BxVnZD86qVFFQALABwEoC6AWwD8N/ZLXkIC4BwA9QEcCeByETkjtcUEwPpM1GIAdwEYkqbyNtiyrttexhJifSbuKQBbYPa3/QE8IyLdUihvMMI/k2QVARgB4JSSzlNEGgF4D2b/0wDABADDIuYVVf+PAjgOZt/zjIhk29fvBXCfqq5LZuU8qlriPwDnApgDQBLMPxjA6xHpeQAKAexcwvcpgPYhaWMADIh57U0A9wSmDwGw1MYdAeQDqB1IHw3gkpDyFwE4PDB9J4C3bfwZgE42fhvA6QDq2EquF1LeAABjkqmPdPwBmArglHTUp83zOIAnWJ+lXo93ARhawveE1guAQwHMS+Z7wPpMqR5rwjSSHQOvvQaz00/k/UMB3JXoZxJ4bRSAC0PK7ANgYUhaFVvfeSWoh4sBjI1Z500AOscpP7L+YX7gFL++FEATAHsBGFGSzyjsL9kxynMBvKp2bgAgIqtFZP8kyzsHwGhVnZvk+xPVDeYXbbEpAJqKSEObNkfdXx5T7OsOEakPYKc4ZRXn/RXAYSJSD0AvANNhviCPqurqNK1L2ohIU5gv4rTAa0nXp4gIgAOC5WUI6zNBKW6fpYX16esIoFBVZwde89ZBRFrbOm2dSGEJfCZpl8A8nfpW0+3/R8gyba/+l4tIDxHpAXOUuwrmKHNgGlal5A2lrZiDYLouPKpaT1WTHew+B6Z1z7RaANYEpovj2nHSitNrY1u1Yt4fm/demIbiW5jukxwAuwL4WETeFJHvROTyZFcinUQkB6av/hVVnVn8eor1ORjmu/Vy6ksYifWZoBTrs7SwPn2R66uq822dzi9BecVlbFNehmxvniWt06i8lwB4DMDzAM4G8C8AIwFUE5HP7djmQcmsBGAOl0vqHJguiLQc/dlfuc0AvLudfJ/BfLkB4J+qmvBAbMB6mC6WYsXxujhpxenx+rbXB9I3x+ZV1b8B9LPLnQXgO5iKvAHm1+wAAJNE5GtVnZ7EeqSFXbbXYLp40rJjsDuYcwAcoKr5EflYnzsQ1mfalWR9Ey2vuIxtPpN4RGR9YLJrBuZZ0joNzauqk2G6hiEizQE8BGBfmB9DV8GM4X8nIm2CPaGJSqbr9RzEHE2m6FwA76nq+qhM6p78k8xGCJiuwB6B6R4AlqnqSpvWNnh2lE3fpvtQVVcBWBKnrHhdjRcD+FFVfwWwC4AJqroFwC8Auie5Himz3aMvwZwocIqqFqShzPNhT8ZQ1YVReVmfOxbWZ9rNBlBFRDoEXgtbh+0q4WdS/J5agb9Ej1xLMk+nvsVcmtQuZJkSrn8AjwAYpKqb4NfpPJieg7CTvyKVqKEUkf0AtEB6zk6FiFQHcBrS2O0qIlVFpBrMWZg5IlLN/moEgFcBXCAiXW3/+aDieduxgMkAbrPvOQmmO2Z4yKxeBTBIROqLSGcAF8Wuh4g0AXAZTFckAMwF0FdEasGMjcxJwyon6xkAXQAcZ79QKRGR/jCXCxymqmlbL9ZnYkSkiv2csgFk23VOpseouLwsW16OmZRqIlI1DcvJ+kyAHa97D8AdIlJTRHoDOAGmByhZ2/1MkmHrs/gynlw7ncg83wfQXUROse+5FcDU4BBQsUTrX0QOA1BNVT+xL80FcLCYs4VzAaxMaiUTOeMncJbQcwBeC0lbD9PdFi9tMOKcHQfgHwD+RMjZs2HvC6Rvc1YdzFlbGvPXJ5B+DYBlANbCjKHlBtLy7Ps3AZgF4NBAWn8A0wLTuTCnz6+15V0TZ/leBXBaYLoVgHEwA80PxeQdgFI6qw7mVG2F6Q5ZH/jrn2x9wnwhC2LKe5b1mfn6DHy2sZ/T4BTqs0+c8kaxPkunPu08GwD4AOZSnfkAzgyktbZ12jrkvUOx7VmviXwmo1DCs17j1KcmOk+Ys6pn2jodhcBZswCeRWAfElX/gXlNBtAm8NohAObBHNmesb3PKLQuSqnCB9nKXg2gZoLvmWW/CEMi8myGGdC9szS/wBn4fL6E6WsfWdbLwvpkfbI+K3Z92mV+wdbPHwnm72DrfyNiLtsJ5DnQNlKrARxR1uuYwmeTa9dhA4DbEnmP2DcSERFRHLwpOhERUQQ2lERERBHYUBIREUWIPH38sKzTOIBZRr4sekcyUS7rtOxkok5Zn2WH2+iOJ6xOeURJREQUgQ0lERFRBDaUREREEdhQEhERRWBDSUREFIENJRERUQQ2lERERBHYUBIREUVI+nl1RKmYd9e+XlxYzb2+unG3v7z4hx5hjxsE2n19nhfXHl/dSWv6+NhUF5GICACPKImIiCKxoSQiIorAhpKIiCgCxyip1Kz6tIMX/7rbkwm9pyDi9tAz+77oxW/0au6k/ffLg7y4cMZvCS4hlRfSs5sz/elHr3nxLs9e7sWt7uRYdFnIrlfXi2c92daLg9skAAxa3tOLf+nf0UkrnD47Q0uXfjyiJCIiisCGkoiIKAK7Xiljgl2tAPD9bm8n9L5nV/tdOQ//cJgX57X5y8n3Rdf3vLh/7SVO2t0DGnlx2/9j12tFs3zPOs70VhR6cY3FfFxjWSvauaUX/9LnOS+OHSq5q8lEL+5x0n5OWit2vRIREe0Y2FASERFFYNcrpdXWQ/yz3L7u8VRMao4XPbrKPwPum3693GyLl3thx1UTvDirWjUn2z3jdvHimxr94i5H/a0JLzOVP6t2LXSmF27N9+KGL/1Q2otT6VVp1dKZ3vn538toScoGjyiJiIgisKEkIiKKwIaSiIgoQqmOUa68aF9nuvXZfj/3zOVNnbQt+f54Vou3/LjGwvVOvqLJ09O5iJSi9S2qenFWzO+w4LjkqOP98cXCObMSKvv323d3pt9s8FBgKtdJazmCvwErGu29mxePPvZhJ+2g767w4vb4udSWqTKbf6t/OUfPI9397APNR5e4vFr7uZd3LbjFL7/RVP+cguofji9x2ZnGvQkREVEENpREREQRSrXr9frr3nSmT6m5yp9oF/HGPn44b+tGJ+mxv/qmvmAJGr+8jRfXfKiuk1Zl5MTY7JVSvVf9U/dPnXCWkyar1nrx1iXzSlz2hUd/5UzXysoNyUkV0d9d/YdvN8+u4aS1eDcnNjtl2NR/PuHFBVoYkTMxo3q84b7Qww/f3+A/1GDIuhOdbFW+Lvt9K48oiYiIIrChJCIiisCGkoiIKEKpjlE+ftMZzvStu/rtdP0Z7m3nV3URL66662ovfqD7e06+R5qP8+JPN9by4mNquJeRRNmkW7x4XH5NL+5TrcDNGJhX+37/dJI6jkx4dpVGOh7MOu9u/5KiC+o9GJPq39Lu30v2cVJqfzXDX46Ul4JKwyGX+uPbH2yo56TVGuVfQsT6zJycUf5YYY5kp1zez1uKvHheQWMn7aSaf3vx6bX821ae/trzTr5jW/REWeMRJRERUQQ2lERERBFKteu15rvjYqbD89YJef2JZn2c6bt65/nv+da/088DfdonvFxVNvndAzWn+g8AbvjdcCffLlUDdwiax9PVM2X12X536/fn+N2tdbPcp4f8kO93DU2+y71rT/W15e/uHuTK7tbJmb6nyVte/NJa92kVhavXlMoyVTabTtzLmT6v+TteHLwkJNHLQ7qPvMSZbjzSv4Qrd41bxo19/OO0X057PLTMhTf6d/Bpee/YhJYj3XhESUREFIENJRERUYQK9+DmrUuXOdM1h/vTwQP7mu+uTKr8ZRf63X7dqrofz4N/+11FeS/PcZcrqblRPCv28M+Aju1uDTp31IVe3PEDdrVWNIsOaxiaNnFdm5hXNmV2YSqRYJf3XQ+7Z5j2qrolmDO0jOCddAZ9c4oXd7l+ppOvcO1ahOn0m/+QhPHH+9v5XrmbnXyf/esBLz682vVOWt49/l17ND8fmcIjSiIioghsKImIiCKwoSQiIopQ4cYoM6FKm1Ze/ORNT3px7J0p3nnsUC9uuOQHUHps+dIdj/qhc/CBzP7YRY8fznXydfn3H17Mu7VUPGu7FoSmTX5yN2e6Hri9pUtR4NwLd0wy3Pl/HulMr+vnP+ml40L//ICSbIfBO3ddOtS/rGTCPx918jXP9uc16QI37ZT3/H2CTpmBTOERJRERUQQ2lERERBHY9Qpg5tUtvHjPXP9m7NO2uKekN5juPjSaklelbZ4X39n+HSetfuCSkImBM77b3Ol27BSuWgWqWPKP2tOLPzz8CSftjhX+za8bDJ/qpBWBSttNy3p58doL3Ut5Chf+ltZ55Q1f4cW3nOg+4OC+Zj+ldV7J4BElERFRBDaUREREESpl12v+MXs605NOfSQw5d/E919XXunkqz6Wd39Jl3b/XeTFu1cN/732j8BNljtOKfsuGErNwoP9Xc6uVd27Lp07bxcvbrLBvcMLZUbUMyen7hF8RnB6u1q3If6QV5Ust6M9ahkX3+7HzU5M+1J5eERJREQUgQ0lERFRBDaUREREESrlGOX8o9zfB7XEH5f8x9zDvLjGiClOPgWlYtW5/pNZbm8avPtOrpPv3Hn+HZC6XO8/jJt336n4Gndf7sWF6o5FVfmwfmkvTqU06181vDjRBzJn2ryT/ctP3m3sngtSoNmB2F3enW7z40xeQsQjSiIioghsKImIiCJUmq7XrNq1vfjsA8Y4aWuL/AeFLr+nrRfn5vNyhFRUabGTM33AwHFeXCsrNza754fp7b244yrWQUVXZWf/pvcPdvLvwvTCmlZOvgZDeOPz0jDogI/LZL5VWrV0ptf19PcPz573dEJljM93LymSLVtTX7AE8IiSiIgoAhtKIiKiCGwoiYiIIlSaMcrfBnfz4k8auf3hJ/x2ihfn/o9jYuky4yZ3DOqDZvHHRvr+cpozzUtCdiy//dMfi9onMDR90aS+Tr5W+LW0FonKwPTbmznT0w5/MqH3DV/fyIufudbdV1SbUTq3FeURJRERUQQ2lERERBF22K7XNWe5D/+c2u9xL/5ja4GTtv5+/7TlXCzJ7IJVIhOPfyTmlfiXhNS91L2nxlY+kHmHUtRqc9zXN62uFvd12nHkjGruxfc2H55UGUMX7efF1T4umyc48YiSiIgoAhtKIiKiCDtU12vwTjBX3TLMScsVf1XPmHK2k9b4M57pWpYKmtZ1pnO2tChxGYV/rXCmNT/fiyXX7/LNbtwIYQob13Omf/t31YTmrYX+Q2c7X/G7k1a4dm1CZezInt779bivt/gs/IG8lDnZ4g91RD0Uee2Z+4Sm3X7HS17ct3r8rvXY8re9AXti9a8HL9p+pgzjESUREVEENpREREQR2FASERFFqPBjlFLFX4Uenyz04tNqrXTyvbGuiRc3vcX9fZDJB37S9n367pCUy9jv53840yuW1fHi+o3XefG4nm+mPK8oXQdd7ky3vb7yPRFj83F7OdP7Vwue0l/hdzkV3n3DTvXi0y94NDTfd/95youjHvBckOAT7RN9SHT3kZc40x0wKbEZZBCPKImIiCKwoSQiIopQ8ftBenTywjubvBaa7al7/Jvp1ptS+brDysIJ0/s70yO7v5uxeY3d/a2k3rdRt3hxgYZ3wh89dYAXr5kcfolJizGl8yDZ8mz+8W5fXPDSrDtW7OLFtT6c6ORLsAePUtR2mH8p1fiz3Lsj7ZUbfqlHqmIfuvz80oO8eNWl/g3TO8+NucQqY0uUOB5REhERRWBDSUREFIENJRERUYQKN0aZ3bWjM33x2x/Gzdd1yGXOdN5rP2ZsmSi+6kfMdaa73eNfOqEJfvNqd/7bi0tyaUe30ef585pfMzRf23fX+xPjfwnNVx+/xY3JyK7jX47zf73/F5rvzc8O9OK2W3muQFkonD7bi2+95kInbcFx/jj97KOeS+t8Lx3iXvbR6u6xgany/cQgHlESERFFYENJREQUocJ1vc68tL4zfVyN+E9naDlqi/uC8uTzsrbzTal1tR2LnonPC1NTmheVTFHgaS3TN+7kpB26qJcXd7hnmheXh9P+K7vqH7oPQu4YGMk68B/+8FXOgGVOvhHd/KczHf7rGV5cNLSJk0/9B+sgb/JfTlpFqn8eURIREUVgQ0lERBShQnS9Bm+yPPK4h2JSa5TuwhDRNoIPyp7Vy02rij+9uCJ1t1V2dd4KXCkQc+Ork+Dvk2tiTiBlDsJU5LrnESUREVEENpREREQR2FASERFFqBBjlIt7Z3tx6yrhY5LBhzPnrHUvD+HFIURElAweURIREUVgQ0lERBShQnS9Rrl3ZVcv/uGIPC/WJeE3uCYiIkoUjyiJiIgisKEkIiKKwIaSiIgoQoUYo2x7g//UiaNv2CMi59LMLwwREVUqPKIkIiKKwIaSiIgogigfaExERBSKR5REREQR2FASERFFYENJREQUgQ0lERFRhFJvKEVkqIhsEZF5CebvKCLrRaRQRC4MydNHRIpsviPTusBpJiKH2uUsEpFDy3p5kiEig0WkwK5HzQTf84et99cj8qiIbBCRu9O3tKVPRL4Wkc0iMqaslyUZ3EblArucKiLty3p50oHbbGLrEyalhlJEOtgdQkln/ICq5sUpr4GI/BXcwajqbFWtBWD0dspcrKq1VHWELUtE5GYRmS8ia0XkbRGpE5hXCxH5UET+FpGFInJJxHo2F5GPRGSx/WLkxaRfJyIrRORXEekeeL23iHwQzKuqX9n1mb+d9ckoEblcRCaISL6IDE2iiGH2895gy+srIt+IyJp4O1hVbQfgngTK7aGqNweW83kRmWV3sgPirMfVIrLUzneIiOQG0vLsMm0UkZlRP0xEJNe+f60t75pAWisR+dF+Vx6Ked8IEekVs64HAwj9PpWmSrSNBhvi4r9zA+mPisgqEflBRFoEXu8vIo8Fy1LVl+z6lCsi0kXMj7A1IvK7iJxUwiJit9l6IvKKiCy3f4ODmVPYZo+z+8L1IjJWRLoG0nJF5BG7L10lIk+LSE7EOh8sIpPs92OOiFwcSOshItPsvvfqwOs5IjJORFoluT7bSPWI8ikAP6VYRtD9AGakqaxzAJwNoDeAnQBUB/BEIP11AHMBNAVwDIB7RKRvSFlFAEYAOCU2QUSaA7gAQFsAzwK4z75eBcBDAK5Kw7pkwmIAdwEYkqbyNtiyrktTecWmALgUwKTYBBE5AsANAA4BkAdTB7cHsrwF4GcADQHcDOBdEWkcMp/BADoAaAOgL4DrxT/yuRHAKwB2BnBiccMoIv0AzFHVCcmvXsZVlm0U8Bvi4r9XAEBE9gLQE0AzAGNg6hMiUhfAtQBuTdP6ZIzdn3wI4BMADQBcDOB1EemYQrGPAKgBs+3sBeBsETkvxeXsAOANmB+K9QB8DOAju/yA2V57AegOoCOAPQAMCikrB8D7AJ4DUBdAPwAPi0gPm+VemPrrAWCQiDSzr18DYLiqLkhlXYKSbihF5AwAqwGMTMeCiMi+MB/ey+koD8BxAF5S1QWquh5mA+8nIjVEpBaAPgDuVtUCVZ0C4F0A58crSFWXqerTiL/DaQ3gZ1VdC+ArmJ01YBrIj1R1XprWJ61U9T1V/QDAyjSVN15VXwMwJx3lBcp9SlVHAtgcJ/lcmDqepqqrANwJYABgugNhNsLbVHWTqg4H8Avi/NixzgFwp6quUtUZAF4oLgumgfxaVdfAfAfa2iOfGwDclIbVzIjKtI1ux84AxqhqPsxnUbyN3g3gP7Zey7vOMD8mHlHVQlX9GsD3MD80knUcTM/BRrufegnJfb5BRwAYrapjVHUrTJ22AHBQYJ6Pq+rfqvoXgMcj5tkAQB0Ar6nxE8yPtOIj1OLtchGA3wC0FpHWMNv4IymuhyOphtLuJO4A8O84aa1FZLVd4ETLy4b55Xs5gHTdAUHsX3A6F+aoQQKvBdO7o+R+B7CLiNQDcCiAafaQ/wwADyZRXrlg63D/sl6O7egGc8RZbAqApiLS0KbNUdV1MendYgsRkfowO6HYsorz/grgMFvHvQBMh2mUH1XV1Wlal7SqpNtoExFZJiJzbfde8VjcNAAHiEh1mN6HabZXoJOqvpmOFSkFEvJacKgnmW02HfvA2PJiywyWGy+9pT26d6jqMpheofNEJNv+UGsD0ysAmO3ycBFpCXNU/AdMw3u9qhakuB6OZI8o74T9JRiboKrzVbWeqpZkDG4ggHGqOjHJ5YnnMwAXihmnqgvg/+zrNezO83sAt4hINRHZA+ZXSI2SzkRVV8L8Mv0apnvoWgCP2fmdJCLf2nGWlqmvUumxdVjeT0apBSB4NFAc146TVpxeO6Sc4Ptj894L4AAA38I0FjkAdgXwsYi8KSLficjlya5EhlS2bXQmgN0ANAdwMExX68MAoKq/AhgO4EeYHqD7YbbRgSIy0NbfG/aHUHk1E8ByANfZMbjDYY7SvM8jiW12BIAbRKS2mJOWzkcS+8AYXwI4SMyYcVWYHpeqgXI/A3CliDS2XaUD7eth830Lpms8H2YM/ObAd/paAP8C8BGAq2G68NcBmGP3ud+KyGkprg+AJBpKEdkN5sgpLYe2IrITzId18/byBt4THLAP+1U8BOZDHgXzi/Ib+/pC+78/zKH7AgDPwPSrL0QSVPUtVd1DVY+C+eWUDzM29iBMV8M7qMBHl5kgIp8F6rB/ksWsh+maKVYcr4uTVpy+DttaH/N+J6/tJuqnqj1gdrBPALgCpuv1V5jt4ZLgSQtlqTJuo6q6VFWnq2qRqs4FcD2AUwPpj6hqD1XtBzPWNRpm/3cxzFHmDJj6LJfsEdKJMD/Gl8L0FPwXSe6zrIEANsF0W34IUxeh5SWyzarqTJghkScBLAHQCKYHprjcu2H2jZMBjAXwAYACmB8BsfPrDGAYzLBIVZgenutF5Bg7rz9V9WhV3cMu/x0wjeeD9n3Hw4xpNkj0AwmTzGO2+sAc5s4XEcD8Gs8Wka52gUtqL5hfgdNtedUBVBeRpQBaqGph7Btiz0gTkbZx8hQBuM3+wf4CW2T/oKp/Ajg2UMabAMYnsfzB5agOc1bVUTDdRwtUda2I/IRyPJZVFuyPilRNgxnI/6+d7gFgmaquFJFpMGOJtQPdrz0AbNPVpqqrRGSJTf8ykHdanHleDOBHVf1VRHaBGTPaIiK/wPxImp6G9UpVH3AbVcTprhSRpgD+CWAfmB+xU1W1wG6jVyZYdplQ1anwx/ogImNhTjJLtry/YX6MFJd3DyI+30S3WVV9F2Y8GfYo/XzY8ztUdRNM9/3lNv1iABPjfYdgtqdZqvq5nZ4lIp/C7F8/jcl7K4AXVXWZ3S4HqeoaEVkIoH3UeiUimYbyeQBvB6avhdko/5XkMnxm31+sH4AzAZwQ8uElxP6KqA9zckkXmG6YO+zGCRHpAvMrJx/A6QAOt/nCyqsGINtO5opINVWNPcFkEIChqrpYRBRAJ7th9kWaT3JJlZiz0KrArFO2Xb+tdgA+mfKyYH715ZhJqQagSFW3pLicVWF++QuAHFvuFluPrwIYKiJvwPx6HQRgKGAuWRCRyQBuE5FBMBvXrgg/medVmDPnJsCcZXkRAOcMQBFpAuAyAPval+YC6Gt3WL1gznIuDyrdNioifWw5CwC0hDn7/MM4WR+GOcFro4jMBbCn+CcOlattNJaI7ApgNsz2cCnMj5ehKZTXDuZkr9Uwn+3FCDTEKZTbE+aIsQHMkeXH9kgTYi7NUZjtdW8At8BcNRDPzwA6iMjBML0NbWF+ON0fM7+uMPXX2740F8DBIrIG5oAl5UvxStz1as+QWlr8B9NttdmewVR8okBUd0tsefkx5a0BUGDjVDQC8D+YyxY+AzBEVZ8PpB8Bs2GsgjmV+cjidbDrsV5EDgjk3wS/i26mnUYgfyeYL9sTdr2WwGys02C6OG5McX3SbRDMOtwA4Cwbe6dpx1n/7TnQlvE/mHGgTQC+SMNyfmHL2g+mAdhk5wU11+M9ALMR/Wn/bgu89wyYBmwVTF2cGvie9rdHncVugzkZ4E+Yscj/2PKDHoTZkRd/D+6FGQ9bAHOGc7m4TKSSbqN7APjBljUWpkt8YKAsiLm0pJ6qvm/XazzMkckCmB+z96W4Ppl2NkwDsxymu/gwNWfyAkhqm+0Jcyb4Opjvcn9VjdeLUlKPwTS+s+z/iwJp7WDqZwPM0fANqurtJ2z37k0AoKp/wByNPg5gLcx2ORzm7LBQ1ZUAACAASURBVNygpwBcGfjRdiNM3U8DcE8avqeAqpbqH8xp9+sB/JFg/g72w94IYEBInuKd9GoAR5T2OpVw/Q+xy7kJQN+yXp4k12GQ/aKvBlAzwffMsvU+JCLPZpid8J1lvY4pfj5fwux8Rpb1siS5/JV9Gz3PLudmAG3LennStE6VfptNZH3C/vg8SiIiogi8KToREVEENpREREQRIs96PSzrNPbLlpEvi96JdyeOlLFOy04m6pT1WXa4je54wuqUR5REREQR2FASERFFYENJREQUgQ0lERFRBDaUREREEdhQEhERRWBDSUREFIENJRERUQQ2lERERBHYUBIREUVgQ0lERBSBDSUREVEENpREREQR2FASERFFYENJREQUgQ0lERFRhMgHN5cXhX338OLLn/+vk/ZMh/YZm++6fvs40/Umr/CXadbvGZsvldzqc/Z1psfd94wXd33qUi9uff94J59u3ZrZBduBVWnTyoubDFvtxd9O7Ork6/y0n1Y4bVbmF8zKbtzYmV55lL+vqD9skhdrfn6pLRNVTDyiJCIiisCGkoiIKEKF6Hr984hcL26Qvb7U5rv0mC3OdMHZ/u+KBseW2mJQiCotdvLiO299MTTf9Mue9uKjHj/ASdN169K/YDuoKs2aOtN3jBruxZ1yirz44JXNnHyF037L7IIFBLtb+4+Z5KTtU+19L77sl3/6CT9Py/hyVXTZjRo607Meae3FfTr49bvooAIn347Src0jSiIioghsKImIiCKwoSQiIopQbscoJaeqFx988OQyWYbaP1dzpk+/4Fsv/qZeSyetcPWaUlkm8i0/oo0XH16jIDTfHhP6eXHj9bMzukw7miotW3hx3WEbnbRdq2Z7caevLvHiDue6Y4OlacZdeV58eq0RTtoej17vxTv9PLa0FqnCWn75fl5825WvOmnH1Pgi7ntObHScM7110eL0L1gZ4BElERFRBDaUREREEcpt1+u6k/y78Tze4gkv7vLB5U6+DhiXsWXIr6/O9MD6M714VO0ubmZ2vWZcVo0azvQRA8ck9L7ct+v7E6rhGWkbq3r7d9/5IO+p0HxdBi334tK815Hu28OZ/v3Y57z4oF9Oc9JaDfG338LMLlaFld2xnRe/+O9HvXi3qm5TUYT4ljxT25lu/k//UqGtS5amvoBlhEeUREREEdhQEhERRWBDSUREFKHcjFFq792c6afuf8yLX1/rXwbQeZB7en8mxxr2PfzXDJZOJZW/nzsufFeTl0Lzbizybz9Y580fM7ZMO5rgE0EA4K8TNofm7fXgFV7cbEHpXW4RHJcc9MYrofnWf+reSq/myjkZW6YdxYwb/PH84OU/iRrX801nevYP/nZ48mvXOGlt7/7Zi4s2h3/PygMeURIREUVgQ0lERBSh3HS9rrrRvetHyyr+SebXXHGMF+esmpjR5ajS3O+uebm1e2ePAuXvirI09+TEu4JO/e3EwNSOcXeQ0rDgsVrO9G97DfXiQcvd4ZEWL/tP3SjNyy0W9anpxb1z3QsVuo8914tbP8G772xPdteOzvRXhzwamKruRfevdIc9Jqz2nx4yrJ27nwzqGLjD2gv9n3HS7h9yghcXzf0zoeUtK9zzExERRWBDSUREFKFMu15XXrSvF7+zy3+ctFfX7OrFOV9ltrs1aPod/ll/Bep2KJ0771AvLlz+V6ktExnH7DklNG1N0SZnumCw/5DhLHa9JkxVnOngNjBuZZ6Tlr1pOTIlq7Z7h5dZd3f14g+Of9iLi5Dj5Gt92i8ZW6Yd0Yq93Acy51Xx73518YIDvXjhPuudfFk1/aGynpf4Zz9fe9F/nXz9a/vfkQPdZ0zg4+HzvXj6MeX7Dj48oiQiIorAhpKIiCgCG0oiIqIIZTpGmXXiCi/eqUquk/bSm0d6cUtk9jTv7G6dvPj1Q/ynD+Sr+zDg+Q/7p1LXzM/cU0vIl3/0nl78ZIsXQvMtjHlkRda3P8fPSEn7X+cPnOkLRvX14vnrmnvxlpfcO+IkaukB/pNdjt7bfVj7Rzs9HZjyxyV7Tz7DyVcfvyU178qq0N3togh+HUx9bhcvboAf3HwbNnhx84f8/fN/j9vTyfeP2p/4E+peyrMs3x+H1s35iS90GeARJRERUQQ2lERERBFKtes1u3FjZ3pQx09D87a8p/TuqjHz0npe3CvXPx3+qVVdnXw1h7O7tbQt2zNn+5kAHPfJVc50Jh/ovSNr8kR1Z/qb5/1z+vtWd29c/VLrb7w4C/5lJUUPJ/dwbKcMhJfx1jr/0p+GNyX2QGGKr/YpS0LT1hzhd682eDmx8m5t81HMK+HHYqN/7uzFHVeNT2wGZYRHlERERBHYUBIREUUo1a5XqeHemuGIGmu8eK+fznHSmmFGqSwTADTK+zvu62/M7eXmw+y4+Shzqu6+KjRtxhb/7iCdH1/hpJXmTbp3JFW+du+C9dj+B3vxnfvlOWkLD/e7R38/7lkvHp/v3t3nrC8uSWjeHV71z3z89J0hofkemH6EF7eYMi00H23fuuHN3Re6+eGArv7wxXd77uVk+2t3/+b5eqy//+ye43ahzijwrxzoFrhBOgC8f9QTXvx/+1zkJ/w4dfsLXsp4RElERBSBDSUREVEENpREREQRSnWMsujv1c70nX/t4cVntpvgpH3XvJ0Xp/tu8lXatHKmv9/t7cCU/9th04+NYt7JMcrSsPlYfzxkwp7Bh726D26eVdDEiwtn/5HpxaqUti5d5sU13lvmpHV8z4+PvmQPhOmIxE79z9rVv1wgeKkIANy1orsXt7nSP7ch5oZMVELNPprrTM++cYsXX9dwuhf/3wfuOSNhl+/0++MYZ3rTQP+SwJPeGuWknVdngRf/MdDf77b7cTsLXQZ4RElERBSBDSUREVGE0u16XbfOmf5ikd/VMnq3N520JZ/U9dOe2xcltbqr2zVQK8/vrtlnp3nucoXcz0OSu8EIpWhTI7+LNUeyQ/NdP/FkL94Z5e+UciqZ+bf5dR3btffF3f5DhGstKId9cxVU7LDWxdf5d7h6+UH/Adkdc2q6bwzc4Lz9F/6lHZ0vn+lkK9rgd9/e9/VxTtoFJ/rDKvf38vvxX+zhdt8WTSm9SwXD8IiSiIgoAhtKIiKiCGwoiYiIIpTpg5vr3+7f0u6gwf9w0t7vPtSL77/NfWhoIibku2NbhYHfBL2qbonJLYin9RO/ONN8MkHpyD9xddzXg7esA4CWLyb2ZBEqn1Zc7J57MHWfp7x43tZNTlr1v2K3WcqEWu/4t607D9d48d+nu9ve5jX+E5+7XOdfmlUYeKBzrE43THemD+ngn2PwZbfhXnzbbe7xW4uTUeZ4RElERBSBDSUREVGEMu16xXi/a7Pu0W7S2X0GevHqDrkoqYYvhHfXLnqvmzM9ce+hcfPFXs5CmZHdsZ0zPWHP14OpXvTZ+u5Ovpyv3CddUMWy8bD1oWmnTr7QmW7yzaRMLw7FCHbD1nonPF+iT+qJ3Z+ufT+wPQd2yffvOtzJ93TzPl6c7ru0JYpHlERERBHYUBIREUUo267XCNmj/K6WhqPSW/amebXdF/aOn0977+ZMy/eT07sgBABY1reJMx12N54nvznMme6AcXHzUcXwXM/XnOklhf6ZlQ0frVHai0OlrPFz/s3y9z7qTC8e19O9S9uV1+Z5cbt/s+uViIio3GFDSUREFIENJRERUYRyO0aZUTE34skK+b3AMcnSsblB/DsjAcDEfP+OLF3uX+ik8aG9Fc/CG/fz4t657iUfP+b745LZvBxkx1fkX1jS8CG/7le85t6VacYZ/h2bjnvzHCdNJ07L0MK5eERJREQUgQ0lERFRhMrZ9RrzQOawBzdT6Why8KLQtI/W7u7FhX+tKI3FoQzq/4+RXhz7cOYLJgzw4jZwH0iQ3bCBP9GkoRcWzvgtvQtIZSLr25+9uM8r1zlp08/3u17X3e12y9Y5zb/UL5N3UuMRJRERUQQ2lERERBHYUBIREUWolGOURdXCxyT/KswvxSWpvCTXfyLMCTtNCc23ckstL9Z81s2OrKjQ/92+/PL9nLRjLhztxR/Mae7F5eGhvpRe7Z9f4Ey/dlozL/5ul3edtCN7nO/FWWMydzkfjyiJiIgisKEkIiKKUCm7Xl8/8llnesYWvyv2H0Ov9+LWGFtqy1TpFPp35Xh+xv5O0lX7zfPiUQvae3ELlM5dOKhszDjwZS8uOtC9dKTbd34XW/vBG7w40YcGU8WxdYF7B67/nnSQF5/91TAnbcV1m724yZjMLROPKImIiCKwoSQiIopQKbte75h7vDO94ekWXtx6OLtbS4Nu9W9pnnfDBiety71ne7FMjnnINlVon9/sd6NNv7G5k/bDuM5e3PmxxU5au6WzvLhw82ZQ5RG8+1K/OYc7aR/v/qIXX7DPpX7Cj1PTugw8oiQiIorAhpKIiCgCG0oiIqIIlXKMEoe4px/XxMKQjFQaCn+f60y3Pq2MFoQyrtrH4734r4/dtPb40Yv5UG6KZ+NJ7mVD48bu5MWrOtX04vo/Iq14RElERBSBDSUREVGEytn1SkREFU7hipXO9PMd23pxffyQsfnyiJKIiCgCG0oiIqIIbCiJiIgisKEkIiKKwIaSiIgoAhtKIiKiCKKq289FRERUSfGIkoiIKAIbSiIioghsKImIiCKUekMpIkNFZIuIzEswf0cRWS8ihSJyYUiePiJSZPMdmdYFTrNE1qe8E5HBIlJg16Pm9t8BiMgftt5fj8ijIrJBRO5O39KWPhH5WkQ2i8iYsl6WRFX27TKKiOTadSgQkbvKenmSwW02sfUJk1RDKSKj7I5gvf2bVcIiHlDVvEB5xRvp+sBfNgCo6mxVrQVg9HbKXKyqtVR1hC1TRORmEZkvImtF5G0RqROYZwsR+VBE/haRhSJyyXbW+QoRmWvLmiAi+wfSzhSRJTa9T+D1diIytnhdSrg+GSUiXewOfY2I/C4iJ5WwiGH2895gy6snIq+IyHL7NziYWVXbAbgngXJ7qOrNgeV8XkRm2R3ugDjrcbWILLXrMUREcgNpeSLyjYhsFJGZInJo2EztznCIrd+lInJNIK2ViPxovysPxbxvhIj0ilnXgwFEfp8yQUQaiMj7dsf1p4icWcIiYrfL0M8khe2yuYh8JCKL7U42L5g5ap42/RBblxtt3bYJm3FU/dty5trttl/g9XoiMklEagfWNd+u6xvbWdeME5EzRGSGreM/ROSAErzd2WZteXuIyHd2n7tMRK4sTkthmz1ORH61ZY4Vka6BtFwRecTW/yoReVpEciLW92BbH2tFZI6IXBxI6yEi00RkhYhcHXg9R0TGiUirYFklWJ9tpHJEebn90GupaqcUyin2QKC8WqpamGJ55wA4G0BvADsBqA7giUD66wDmAmgK4BgA94hI33gFicjeAO4DcCqAugBeAvC+iGSLSBWbtgeAKwA8GXjr4wCuScO6pJVd5g8BfAKgAYCLAbwuIh1TKPYRADUA5AHYC8DZInJeiosKAFMAXApgUmyCiBwB4AYAh9j5tgVweyDLWwB+BtAQwM0A3hWRxiHzGQygA4A2APoCuF78o6AbAbwCYGcAJxY3jHYHO0dVJyS/emn1FIAtMN/p/gCeEZFuKZQ3GOGfSbKKAIwAcEpJ5ykijQC8B+AWmO/tBADDIuYVVf+PAjgOwJEwn1Pxj9l7AdynquuSWblMEpHDANwP4DwAtQEcCGBOCuU1gqmL52A+o/YAvkhxGTvA/KC4BEA9AB8D+MjucwCzvfYC0B1AR5j95qCQsnIAvG+Xry6AfgAeFpEeNsu9AK4F0APAIBFpZl+/BsBwVV2QyroE7chjlMcBeElVF6jqepgvWD8RqSEitQD0AXC3qhao6hQA7wI4P6SsPADTVHWimutpXgXQCEATmC/YIlVdAuArmJ01RORU+3qaHyGaFp1hfjw8oqqFqvo1gO9hflgk6ziYHzsbVXUezI+JsM8zYar6lKqOBLA5TvK5MHU8TVVXAbgTwADAdA3CbIS3qeomVR0O4BeE76DPAXCnqq5S1RkAXiguC6aB/FpV1wD4CUBbMb0TNwC4KdV1TAcx3WmnALhFVder6hgAHyG1Oo36TJKiqstU9WmYz7Gk8zwZZjt8R1U3wzSqPUSkc2whCdR/TVX91W77WwA0FJG9AOysqv9NZR0z6HYAd6jqj6papKqLVHVRCuVdA+BzVX3DHjWvs595Ko4AMFpVx6jqVpj9bgsAB9n04wA8rqp/q+pfMAcTYfuJBgDqAHhNjZ8AzABQfIRavF0uAvAbgNYi0hqmjh9JcT0cqTSU99pD3u/F7W5sLSKr7QKXxKW2a2uiiITtzEpC7F9wOhfm16oEXgumdw8p6zMA2SKyt/3leT6AyQCWAvgLZiNrCeAwANNsQzwI5kikPJKQ17z1t3W4f5x8iZYb9XmmSzeYI85iUwA0FZGGNm1OzJHBFPu6Q0Tqw/xwiC2rOO+vAA4TkXowv4anwzTKj6rq6jStS6o6AihU1dmB17x1KOl2mcBnknYJzNOpb9uF+EfIMm2v/pfbrrseMEe5q2COMgemYVXSzu53egFoLGaoZKGIPCki1QN5SrrN7gPgb9s9ulxEPk5iv73NomLb/UBwXxAvvaWI1I0tSFWXwfQKnGd77/aF6WkoHvv/FcDhdt+bB/NdeBzA9apakOJ6OJJtKP8P5sipBYDnAXwsIu0AQFXnq2o9VZ1fgvIeh2nAmsB0qwwVkd5JLluxzwBcaMcp6tplBoAaduP5HsAtIlJNRPaA+RVSI6SsdQCGw1RQPoDbAFxsf+UUAfgXzBHptQAuAnAHTDfvLnaM5HMRyXSjURIzASwHcJ3tzz8c5heft/62DktyMsoIADeISG0RaQ/zYyLs80yXWgDWBKaL49px0orTa2NbtWLeH5v3XgAHAPgWpnszB8CuMN/7N+0Yz+XJrkSaRK5vEtvl9j6TTNjePEtap1F5LwHwGMz+62yYbXgkgGp2e/1GRA5C+dEU5nt3Ksx3cTcAuyPQbZnENtsSplfmSgCtYYai3kpxOb8EcJCYE7mqwvS4VIW/L/gMwJUi0th2lRb/MAnbV7wF4FaY/e5oADcHulSvham3jwBcDTPMtg7AHDHnn3wrIqeluD4AkmwoVXWcPUzPV9VXYBqdo5NdCFWdpKorVXWrqv4Ppo/75LD84p70E/YLaAjMhzwKwDQA39jXF9r//WEO3RcAeMbOcyHiuxBmx98NptLPAvCJiOxkl3+kqu6jqgfB/DrtBWAogNdguo3uBPBi1GdQmuyvrRNhxmaXAvg3gP8ifP0TMRDAJpgukA9hPvvQ8kTks0Ad9k9ynuthumaKFcfr4qQVp8cbe1of834nr+0m6qeqPWB2rk/AjEffAPOr9lAAl0jgpIUyUJL1TbS84jISKi/B7TKVeZa0TkPzqupkVe2jqnvD9BCcD3Oix4swXZznAXhNROL1vpSFTfb/E6q6RFVXAHgYKex3bZnvq+pPtiv7dgD7xTu6AxLbZlV1Jkzj+ySAJTBDVNPh7wvuhhk3ngxgLIAPABTA/HCPnV9nmDHoc2D2u91gxqyPsfP6U1WPVtU9YPY5d8A0ng/a9x0PM6bZoGQfy7bSNUapiN+dl5HyYk76ifsL2fbh36aqearaEqaxXGT/ij/kY1W1sd1YGgIYHzLLHgA+VnOmX5GaM/iWANgvmMluVE/CNBqNAGSr6p8w4zG7Jr76maeqU1X1IFVtqKpHwPQQhK1/IuX9rar9VbWZqnaD+W6FlqeqRwXqMNmzCafB1E2xHgCWqepKm9ZWAmcv2vRpcZZlFUx9xpa1TV6YE59+VNVfAewCYIKqboEZ/yrLXoPZAKrYkymKha3DdpXwMyl+z3a3yxTn6dS3HZdtF7JMCdc/zHjWIFXdBL9O58EcwYWd/FWq7GezEGbfmC5TY8orjuPuexPdZlX1XVXtrqoNYXrf2sCOSdvx4stVtYWqtgWwEsBEjX/CY3cAs1T1c7vfnQXgUwBHxcl7K4AXbXdtcR2ugfnM2od/BIkpcUMp5vTpI2yXZRX7y+JAAJ8nuxAicqqI1BKRLNsNeBbM4XTSxJwq306MrjC/vu6wXaXFl0fUFpGqInIWgMNtnnh+AnCMiLS15R0GMyb0a0y+CwH8rKqTYb4A1e28+yKFs9MyQUR2tXVYQ0SuBdAc5ig42fLaiUhDO5ZwFEyDkvI1Z7Z+qsFsvDl2mYu/t68CuEBEutrxrUGw62DH6iYDuM2+5ySYHyvDQ2b1KsyZc/XtL9mLEPN5iEgTAJfBnEQCmK6qvmLGpHuhDOvYjte9B+AOEalphy5OgOnVSNZ2P5Nk2Posvown104nMs/3AXQXkVPse24FMNUexTgSrX+7LVdT1U/sS3MBHCzmbOFcmO24vHgZwBUi0sR+36+COXM9lfJOEpHdxJxheguAMamOu4tIT7sfaAxzxurHxXUk5rK8nex+dB87z9tCivoZQAcxl4iImOG9Y+GOYcPuY/vA9AwCfh02hRnSK/GPtm2oaon+YH5h/QTThbEawI8ADgukt4bp9mgd8v6hAO6KeW00zPjBWvshnBHnfaMAXBhSZh8AC2Ne6whgFoCNAP6EuUwjmH4VzIk4G2DGHnvFpK8HcICNBeawfr5d7xkAzo7J3wim4awTeK0/TNfmPAB9E12f0vgD8B+YExjWw4wbtA9b/zjvHQzg9ZjXTgew2H7ekwEckcj7YtI1znKMsq8H//oE0q8BsMx+d14GkBtIy7Pv32S/C4fG1M20wHQuTHf9WlveNXGW71UApwWmWwEYZz/Hh2LyDoDZ6ZRmnTaA6craYL+rZwbSktkuE/lMQr/HiLNdBurZ+Ut0njDd3DNtnY4CkBdIexbAs4nUf2BekwG0Cbx2CMz2ugQx+6F4n1Ep128OgKdh9rtLYc7tqBZIL9E2a1//F0wv2yqYSzlaJfK+mLqM3WbHwOwn/4ZpKGsG0g60n+9GWyf9Y977GYCbAtOnw+xX18EcHd4PICvmPd8A2Dsw3QOmu3dFnO9P5PqErmcZVPYLtkL/SDB/B/vF2AhgQEieA+3GsBpxdtDl6S+R9SnvfzBHbhvsetRM8D2zbL0PicizGeYH051lvY4pfj5f2g17ZFkvSwmWuVJvl9tZ11y7DhtgLjcp82VKYh0q/TabyPqE/fExW0RERBF25BsOEBERpYwNJRERUQQ2lERERBGqRCUelnUaBzDLyJdF72TkQmfWadnJRJ2yPssOt9EdT1id8oiSiIgoAhtKIiKiCGwoiYiIIrChJCIiisCGkoiIKAIbSiIioghsKImIiCKwoSQiIooQecMBIiKqnLJq1PDinmPXOWm3NZ7sxYdPP9mLqx72Z+YXrAzwiJKIiCgCG0oiIqIIbCiJiIgicIwSQJVmTb14S4edEnpPzuxFzvSsG9t6cb3p/n11G8zY7OTLGv1zMotIVGFsPm4vZ7r6Z5O8WHt19eK5x9d08h1w8C9ePPrrXULLb/5DoRdX+3h80stJ2wqOS85+vpMXf9D4eSdfUSBeMKW5F7cDxyiJiIgqHTaUREREESpN1+uas/bx4pVHu92hN+w+wovPqfO/hMp7aU1rZ/rk2u97cf3TqoW+79gWPRMqn6i8y27U0IsLh1X34rc7POzkW1aY48V1s0Z5cesqNRDq3O9Ck5aftdGLFz9e1Un75z1XenHDF34IL5/imnNzDy+e3vdxL+4/5ygn38q7d/bidiN+zPyClTEeURIREUVgQ0lERBShwne9ZvXo4sUzr/DPoht9+KNOvsbZP/nvScPvgwvqzo95Jby7lWhHNPsxf/hhVueXAilul2qTbD9+enVHL560zh2+WLihXui8ssU/z/LTTh/HLRsAhg36jxdfMuNyJy1rzGRQtC1NtsZ9feroDs70ziMqV7c2jyiJiIgisKEkIiKKwIaSiIgoQoUfo9ywc20vnn3UM4GU6ttmTtGzq/2777zx555JlVEXv6drcSqFrN38O7lsbubeyWXeif4dkE7d6ycnrUD9watvXvPvFNP82zVOPv15WlqWszLQfXs408P2ey4w5e9KRmxyxyjvu+5cL649bYWf8NffTr6sVQvC553l12fHhy714umnP+Hka5dTy4s3DVrrpNUd4N+Ba+vSZaHzqsxyam3x4nVFftz6y/yyWJxyg0eUREREEdhQEhERRSg3Xa9VWrZwpmf8X0svbjrW72Kr85Z7F4isfPXi2QV+V8GCre6p5q2qrPbiAb+e66StmuHfYaTpT3559ca6XUG6fr0X113NLtR00d67OdNzLvPjN/d9wYt7Vo25FiBR1/k3zt507RYn6fnVftfu01MOctI6XDDDi4s2u3dzqowK6rp3wdmtqr/7KIK/3Vz38vlOvlbvj/XiQiSpyH9n+6v9fUCXqu4lIFNPeMyLv93lXSet96F+l23d19n1CgDZ7Xd2pqcdOMSLr1x8iJ/vm0mozHhESUREFIENJRERUQQ2lERERBHKdIwyu15dL97r07lO2geNPvLi3hPccYig3M/8ywKuO2aAFxdOm+XOq4t/C6YGs/5w0hoUzY5bdvybOVGyivb3xyLn+cNF+LT3U06+dlWCl/b445JfbnIv+blp+olevHq+Oyb964n+ZQO3LPOfHPNAswlOvh7V/QfNPrzXMCftxqsHeHHLe8eisiusJqFpu44d4MWt7y69z6rDZeOc6U8O9R8ifFqtlU7a6uM3eHHd1zO7XBXFrMHhtw0sTflH+ZfbrWsV3iw1nuhe8qMTS+fyLh5REhERRWBDSUREFKFUu16zqrlP2Mh/1+96vanR105ap/f8vrnO7/uH11Gnl8d2tzppM35LcCkpXea86V728UbopR5ul+o/5h7mxT/N9E9f73zlDCdf4w1+fTeOmfclPQ/14uUD23jx1c+4l5gMajrKi0dvau6kEy/kmgAAIABJREFUTb7c77498fUTvHjrgoWojDrdGN7NlT2xdmhaabr5J787/rS+Lzlpl3XzHwb9CeqX2jKVZ4/sPSw07fs39/DiZki9O/2PN3Z3ph/b+y0v3qXqGC9ump0bWsbvBe6A2AnvXu3F7a7N3AOkeURJREQUgQ0lERFRhIx3vWbX97s4Zt7Z0Umb1eVpL54Yc8/dznfM8eLCte6ZTlR+ZNV0b1T+2x27ePGMg9yzWbMCZ7D+FLijUv8PL3Pydbrd72LtuNo/S7UIidul9iIv/rKK33074T89nXwNH/bPmjyx5mq4ws/yrCyydu3sxX3qfemkzS7w71bUaGpBqS1TlPrfBoZ3+pbdcpRn2XXqeHHNLHfH+8Umf3tu9khi3a2S49+xaUvfXZ20m5952YsPrDbRScsRf38wPt/vbj1n5mlOvmt2/sKLj6+50Ul7+kS/e/3RISd5ceH0+FcyJItHlERERBHYUBIREUVgQ0lERBQh42OUi8/q4sWzTnIfsvrRBn/88qVjD3PSCv9y755D5dPq43dxpr8+7UEvzoL7AN+Rm/xxiPsu9Z/g0v4L97TuRJ8wIVX8r29Wp3ZO2osfNPDi/7z6ihfvUnV5TCn+MmaL+7txl3FnenGL5ZXz+/jbuf6dW86o9ZeTtv/Us724zv/cB2dT+TX3qu5evH+1kU5a12/O8eL2+Dm0jOBTR2Zd5j8QO/ZB2kEjN9Vypi/9fIAXd37Mf6B37mx3W3sK/rktT4xs5aR90vk9L763tX+5YdXpoYuRFB5REhERRWBDSUREFCHjXa/r9t4UmvbYXP/BoNVnV86urYpOY56lvFnDL6lYV+TfgWfp3v4p5ZtO3svJ177DkrjvX7PZvbPTaW38h8leVu81J23CFr/83rnBC0vc7uCg7ze7F6C0uMtfF83Pj81eKVx91KdeHLwcBACqPtUwMMXtt6KQXcMvt8v5o3poWlDwZuoz+/qXgcVewtV/zlFevPb6Fk5ahx/8S7MSHW75fU4z94XO8fOlG48oiYiIIrChJCIiipDxrte3ej8fmHLb5Xe7+g+F2/fhfztpO3+0xYuzR00ClU/1P3RvlH3xOf29+PXO7kP/jq/p343nlH/5d2Uq1PB77uSrfxPkXIn6urppbnerb2tMJ0+fqWd4cYPL3DSdUzrPuqsonlt5oDNd7ZPxZbQklIrOTZaV+D3Ss5sz/f7+zwSmcryo26iLnXwdLvDvsiWbp5R4vttz63L/OZbVRv3ixSW5i1cieERJREQUgQ0lERFRBDaUREREETI+RrlXrt9/XaDuGFD9LP90/5n93CdNFJzu5+0+8hIvrvuTe4nA+pb+uFcd/4EjaDR1Q+gyrdjVfeJF01H+3VoKeZlKiRStW+dM5x7uT1/c9GQnbcbgPC8+vKc/njB7TRMn35+LGnlxdlX/e3B8p6lOvgeaTUBJdf3GHUPp9G//KSNbl8Xetafyya5X15munVU5H1K9I2tZw39KTlbssZIo4pk90H2Ycpccf7/e86ezvLhdf/duPukeK8yptcWZ3rDVX66izZtjs6cNjyiJiIgisKEkIiKKkPGu150/vsiLZx/7bMLvCz7Uc9ahL/gJh6ZlsRzjb/DvwHLV9MDlAsem9+GflU1hTFdmx3/50/MCr1fFn06+DjHTxb54v6szHdX1Om+r/4DXE5+43i/7UfeShsKtW0G+hRe4lwH0r/2NF0/akFfKS1Ny+UevCU3bWFQ1NK0yKVL/+KgotnM05M5azZu6DzUPvq9rY/9yk1VpWL5YwRuwTztwiJN24NTTvbhOBu8OxSNKIiKiCGwoiYiIIrChJCIiipDxMcpOl/mnCx/xjntq/jlPfuzFNbLcpzMcW8N/SGxwvDIT9sr1T4kes/sbXtztPwOdfO2u+yGjy0HbmnvPvl48ac9HYlLDx5xOfcAfl9zpqbFeHP/kd6qoth7c05l+e/cnA1PuJQ3v3+8/ragu3IeFU7R6F7iXXowb7V8e8mRrfz++7/3XOvk6Pu6fb7B10eKk5t1lmF/GskL3aVTVHmsQmOIYJRERUZlgQ0lERBQh412vGjj9PueriU7aW513Cn3f46f6l2kU5vinLO93rXt6/33Nfkp1ER3BO1W07BH/AcKUWYuv28+LP+//gBdXl/CHLj+2qr0z3ezlyV6c7ruDUNkKdrf+faV7B67OOX5366WLejtp9Yb5TyGqTF3wwcsrAODAul+XuIzYbtP7Dz3Ri3sM92+J9utZjzv5Lj2orxcvOaaBk1a48m8vXn22P8Sy/1XjnHy3Nv3ei3u+7XbtthtROl3oPKIkIiKKwIaSiIgoQsa7XpNV891xcV//uMe+zvR9Z/tdrxvVv2Fuz+/+5eRr86J/5uyKgRudtAl7ug8YptJVcHgvZ/qDy/3u1tZVwrtb5wfuvvPR/x3ipOVuTG+XfGVRZ5774ILgHY7KklTxd1Wrr/ZvvD9hj7edfF9uqu7Fs29x7zJUtaDkN9HfERT+PteZfnvpXl58UrsRTlqb/ed7cXadOn4Za9c6+bbOmefFE3f3j7cOPNu9UqDBVP+OPtKowEmb+2QrL552oH+2cuyZrcHu1nbXls3ZyjyiJCIiisCGkoiIKAIbSiIiogjldowyTOvP3Tv44Gw/rCH+nVpmHPSSm63NYV78v7zPY0qN/3th/lL3dOYOzjMvKF3mHeveeSkvZFxySaE7XnbOVf/24hqfxh/TppKpOdz9HEfc2cWL21X7y0n7rWV3L966cBFSVbT/bl4891I37ZQu/uU+9zRxxyWD7rn2XC+u/vn40HyV2eYL/bHHh4d3dtI+6fyhF1850r+8Zvyz7rkhtRbHf+rOX3u6F2PtOdC/dOShncY4acFL8Z5fk+fFQx881snXbkjZ3xGNR5REREQR2FASERFFqHBdrzkTfnOm95n0Dy/+cY+3Qt/3Wt6XgSn390G++qctHxt4cHPnge5Ndt0T5ykV2Q39bu2fT340JjUX8fQZc7kz3e59dreWpkvruZcZLPvE78Kb8HfrlMu/b+fnvXi3quG7polb/C3x7PEXOGntvp7pxdxe4yuc7e/XvjvBvYSm/qf+nY4e2Wm0n3DHaIQJdqFu8yDoCN3HnOfF7a9Z4cUNFpV9V2ssHlESERFFYENJREQUgQ0lERFRhAo3Rlm0bp0z3eyK+l583JDjvfimvE+dfPvm+iMWw9c3ctJu/l8/L25/tX+LJI5xpFd2fb+urhrnj3nUkvhjkgBw/0r/8oQOF7nj03wqSOYFT9VffuV3Ttrtjaf4E8E4af7uaGvM1jfFvzslzhrm3yZt5xvc8SxusyUTvBUdAHzQx7/k5/Hz/CeEbNjZvf3c50f65xUc8flVfkLEY1k6veg+/Dnvp6n+ciSysGWIR5REREQR2FASERFFqHBdr7G2zvPvdo+D/XDgQPfWHuv29O9I33nQCiet/Z9lc0f6ymbF8f5dQA6v8Y0XF0Z01/zv9j5eXHMDLwcpbQ0Cd0X56buOTtrDH/hdadfUd7vFk9H52/O9uOov7t2ZWt471ot3Rvm7fGBHUbhsuRe3uG95aL4r4N+1pyMSe1JPRX5YNo8oiYiIIrChJCIiilDhu17DNH18rDsdiMv7GVY7qlOu/cqLCzX8nNX2H1/ixR2Hs7u1vIh9APBX3Wv7MfZIufy2mLz9TERlgEeUREREEdhQEhERRWBDSUREFGGHHaOk8qdHdf9Snmzxf6P9uNm9n0rXB/zT0jmeTERljUeUREREEdhQEhERRWDXK5Waq97wH7I786Knvfj8IVc4+VrNcS/tISIqSzyiJCIiisCGkoiIKAIbSiIioggco6RS0+Y2f+zxiNt28+JW4JgkEZVfPKIkIiKKwIaSiIgogqhW5MdpEhERZRaPKImIiCKwoSQiIorAhpKIiCgCG0oiIqIIpdJQishgESkQkfUiUjPB9/whIltE5PWIPCoiG0Tk7vQtbekTka9FZLOIjCnrZUmEiAy1dTMvwfwdbd0XisiFIXn6iEiRzXdkWhe4FIlIrl2HAhG5q6yXJxHcPqNVtO0znsq+zSayPlESbihFpIv9wqwRkd9F5KQSzmuYqtZS1Q22vHoi8oqILLd/g4OZVbUdgHsSKLeHqt4cWM7nRWSWrcABcdbjahFZatdjiIjkBtLyROQbEdkoIjNF5NCwmdod4hARWWvLuyaQ1kpEfhSRv0XkoZj3jRCRXjHrejCASxJY17QRkQYi8r7dkf0pImeWsIgHVDUvUF7o56Gqs1W1FoDR2ylzsf2OjLBlNheRj0Rksd3p5gUzR83Tph9i63Gjrdc2YTOOqntbzlwRWSIi/QKv1xORSSJSO7Cu+XZd39jOuqaViFwuIhNEJF9EhiZRROz22dd+Hmvi7Vy5fZY+ERllG+z19m9WCYuI3WaLG8/1gb9sIKVtVkTkZhGZbz/7t0WkTmCeLUTkQ/vZLxSRyM9VRK6w295a+/3eP5B2pt0m54pIn8Dr7URkbPG6lHB94kqooRSRKgA+BPAJgAYALsb/t3fnUVYUZxvAn5dtYAARRgGRZVhFkE1ARAMCSlwiCBGjCBo0SFCiSVCiURREE83niRviEgUVd1FHxBNXlhgXRJB9S2RTEIEgu+zzfn9UT3XX5XbPnbvMDMPzO2fOeftW3eq+3dO3blVXdwEvikiLZFbqeQhANoBcAGcAuEpErkmhvAILAdwA4OvYBBE5H8BtAM711tsEwN2BLK8AmA8gB8AdAN4QkRND1jMWQHMAjQD0BPAn8X9V/RnA8wAaA+hXcOJ5X7KrVXVu8h8vbSYAOACgDoBBAJ4QkdYplDcW4fsjWfkA3gdwaVHXKSInAHgLwJ0w/7NzAbwWsa6oY/8wgD4ALoDZTwUn4H0A7lfVXcl8uDT7HsC9ACalqbw9Xlmj0lReAZ6fqfmdVzFVU9VT0lDe/wXKq6aqhwt/S6SrAVwF4GwA9QBUATA+kP4igDUw3zu/APBXEekZryAR6QLgfgADANQAMBFAnoiU9+qk+wGcDuBGAI8F3voogJFp+CxWoi3KljAf+iFVPayqMwB8BrNDktUH5iD9pKprYXbCtSmUBwBQ1QmqOh3AvjjJvwYwUVWXquo2APcAGAKYpjnMTh+jqntV9U0AixH+JX01gHtUdZuqLgfwdEFZMCfgDFXdAeArAE28X1W3Abg91c+YKjHda5cCuFNVd6vqpwDeQWrHM2p/JEVVN6nq4zD7sKjr/CWApao6RVX3wXxxthORlrGFJHDsq6rqElVdCPPjIkdEzgDQWFVfT+UzpouqvqWqbwPYmqby5qjqCwBWp6O8QLk8P8u2PjDH8DtV3Q3gbwAuF5FsEakGoAeAv6jqQe98egPh3/u5MOfwPDU3/E8GcAKA2jA/ljao6kYAH8P8qIKIDPBen53OD5VoRSkhr51mF0S2B5vFSZTrlJchrWF+0RZYCKCOiOR4aatjWgcLvdcdIlIT5odDbFkFeZcA6C0ixwPoBGAZzEn/sKpuT9NnSUULAIdV9T+B1+z2i0hD73g2TKSwBPZH2iWwTudYe12Kq0K2qbBjv1lE2olIO5hW7jaYVuZNafgoxSLJ87O48fws3H0i8j8R+Symu7FI52zADV436DwRCfvRURSCI7/Xs2Ba9xJ4LZge9r3/HoDyItLF68W5FsACAD8A2ALzg7U+gN4AlnoV8WiYHoO0SrSiXAFgM4BRIlJRRH4O4ByYrlMAgKoe77VMEvU+gNtEpLqINIPZCdmFvCdV1QDsCCwXxNXjpBWkV8eRqsW8PzbvfQC6AfgXTBdnRQBtAUwTkZdF5BMR+V2yHyINIj+rqn7rHc9vi1BeQRlHlJchha2zqMczKu9wAI8A+AdMq/t6ANMBVBaRD7zrZuck8yGKSxLnZ0ng+RntVpiW08kw/4vTRKQpkNQ5C5guyuYwLbQ7ATwnImenuI3vARgq5npyDW+bASDb+5HzGYA7RaSyiJwO0yMQ9r2/C8CbAD4FsB/AGADD1MiHOQ/fAHALgOsAjIPp5m3jnZMfiEhaGl8JzR6iqgdFpJ+3EbfCXO953dv4ZN3klfdfmO6iVwAMDMssIu/B/HMDwG9VNZnBErsBHBdYLoh3xUkrSI93/Wl3IH1fbF5V/RHA5d52lwPwCcyX7W0wv2aHAPhaRGao6rIkPkeqivJZEy2voIwj9kc8IrI7sNgqA+ss6vEMzauqC2C6jCAiJwH4O4CuMF+0f4C5PviJiDTSY/SZkDw/M09VvwwsPi8iAwFcBPcaYFHKC14n/qeIvARzyeKzePkTPGcnAWgAYBZM/fJ3mO7Y9V76IJgfJ9/BdOu/FFHWUJgGVGsA3wD4OYB3RaSDqn7vdeFP97atLUzvwCgAawH8zNuOZwCcGVJ+whIe9aqqi1T1HFXNUdXzYX7ZzEl2xar6o6oOUtW6qtra25bQ8lT1wsAF52RHFC4F0C6w3A7AJlXd6qU1kcAIRi99aZxt2QZgY5yyjsgLM/BptqouAdAGwFxVPQBzfSXTXc1h/gOggog0D7wWtv2FKuL+KHhPcABBUX4FJ7pO51h712WbhmxTwsceZhDaaFXdC/94roVplYQNLCnzeH6WCEX8y2IZKS+Rc1ZV81V1jKrmqmp9mH2+wfuDqq5T1YtV9URV7QJzrTHse78dgGlqRqzmqxlZuxHAWcFMIiIwg3lugrmGWV5V18Fcf26b+McPV5TbQ9p6zeVsEbkFwEkAnkt2xd4Q3hxvBNOFMP+wKd93JiKVRKQyzAGv6G1zweecDOA3ItLKu44xGt5n8K7XLQAwxntPf5id/GbIqiYDGC0iNb0BItchZn+ISG0AI2AGkgBmtFdPry+9E9I8UCJR3vW6twCME5GqXnfLJQBeSKHYQvdHMrxjWXCLQJa3nMg68wCcJiKXeu+5C8AiVV0Ru45Ej72I9AZQWVXf9V5aA6CXmNHCWUjTQJpkiEgF73OWh7muU9kbGZhseeW88iqaRaksIpXSsJ08P5Mg5lak8wuOq4gMAtAdwAcplDlARKp5x/rnAAbDDOpLZTtred/tIiKtADwIYJzXVVpwm2F17/9gMEwr8cGQ4r4C8AsRaeKV1xtmfMWSmHxDAcz3en62Aqjirbsn0nUMVTWhPwAPwAxi2A3TD90sJn03gG4h7x0L4MWY134F02X1E8wJcH4i74tJ1zjbMct7PfjXI5A+EsAmADsBPAsgK5CW671/L4CVAM4LpA2CGYFVsJwF082w0ytvZJztmwzgssByAwBfevvx7zF5hwD4NNHjkeofzC0Tb8PcBvAtgCsDaQ2949kw5L3PAbg35rVE9scsAENDyuwBYH3IMXb+El0ngPNgrq/v9dadG0h7EsCTiRz7wLoWAGgUeO1cmG6ejQCuKGwfZfh4jo2zr8YG0ot6fvaIU96swt4X59jx/EzP8T0RpuLYBWA7gNkAegfSkzln/w1z7XYnzGCnK+K8bxaKcM7CVGQrYb7X18Xud5hLFVtgvnc+BdApJt3+n8L8mBoH8/20C8ByAFfF5D8BpuI8LuZ/4QeYc7Nnop8ncv8X00Ee7e2Y7TBD7RN5z0pvp02KyLPPO9D3lNQ/cJr2z0feP8L0kt6WBLf3ae/YrEowf3Pv2P8EYEhInu7eF+B2xPnRdLT8eV/Q273/9zElvT0JbjPPz+jPelSdnyGf4Zg+ZxP5PFF/nI+SiIgoAh+KTkREFIEVJRERUYTIUXG9y13GftkS8lH+lHQO+7Z4TEtOJo4pj2fJ4Tla9oQdU7YoiYiIIrCiJCIiisCKkoiIKAIrSiIiogisKImIiCKwoiQiIorAipKIiCgCK0oiIqIIrCiJiIgisKIkIiKKwIqSiIgoAitKIiKiCKwoiYiIIkTOHkJElE7fPHSmjVdd/qSTdvW67jbe1HVnsW0TFd2hXh1tvKa/X43cfO4/nXzDaqy1cTm4E3Pkw58kZczmDjaetvY0J1+9+8r7C3MWJ7W9qWKLkoiIKAIrSiIiogjseqVSp0LdOjbecXaujTf0duezXdP3HzY+qIedtLMXXGHjLd/VtHGr+39w8h1a+21K20pFc/aZy0LTJjf6xMbd+v/WScvO+zJj23Qs23DrWc7ynuYHbDyw45zQ991d2z/38pFv43Ixba9g2qmzhjlptd/JsnH112bbuB7C/0dKCluUREREEVhREhERRWDXK5UIyfK7XVbffbqT9tiAZ2x8TpWfQss4qP7vvGAXDwD8u/3L/kL7QJhzrZOv4WUJbS6lSbB7Ncr33d0Rks3yMrE1tPCmx5zl4EjUTYf32vjxrW4XbYv3/K7xqv+tZOPK/3Mvj+RM/MLGTTE/tY0tQWxREhERRWBFSUREFIEVJRERUYSj/hrl4R7+9a0Kd22y8bRT3nHyVRT/6Q5RtxLk3FHRxrJ2g5Nva59WNq719hInLX/XrqJs9jHv21H+kz0WX/VIUmVcs+5cG09s9FFC71lw1iRnuS86J7Vuyqxmf5xdeCZKWffFA5zlGW1es3HwuuS8Dm6bqgXmZnbDShm2KImIiCKwoiQiIopwVHS9Bm8l2NW3vZM25j6/Ky14K4F7swBwMDBqOepWgtPvHGLjdnXd3xFTc/2h1J2Pv9FJqzP+8/gbT5Z2bWfjSdeOL/L72z57k7Pc+J6vbdzyoRFO2opLJhS5fKJjzfHXHXCW352eY+N+x8+z8YJTr3TyHV7+38xuWCnDFiUREVEEVpREREQRWFESERFFOCquUe7v0cbGMx5+LDTfzL3VbHzXve6jyir+pLHZrZ2N/N8LlQJPTPvTLe6tBDvyD9m42kb3FhM6UvCaJADovT/auKN/2fmI68l5u2vbeNKQvjbO/dKdzUDz/WNwyh8XOmkXvn29je950p/poFOWe9zOW+Lf1vPxadVjPwKlWdPXhts4duLmoOAEzwBvF8mUQ9+td5Zvyxtk42WD/e/aA3Xdc6P88sxuV2nDFiUREVEEVpREREQRSm3Xa7Db7r4nngrNN3DVRTbeOaaBjWvO/CJe9rhqNGts4/ZTVtn41Eru74iWU/9o4xZvcCLZwmzuXNVZ/qql35UdfFLSjnx3iPqY1/0nJeV+kdhx1P37neWKH/pPDhn8gd/dt7SP23U/qpZ/vJ9+5ddOWuOBbncupS6qu5VKgcCkLeUCC1tbV3ay1ZKOSETWXP82ksM7d6a2bSWILUoiIqIIrCiJiIgilNqu1213+JOGBkdIXrTil06+8rcc58fzv0YytnesY+MxtV8Pzdfgw6SKP2aVO2+rsxx8IlLwSUnXrO7r5Mu9M/Fu80S0uN4fLTv+Z62dtJG1Vth4UKuvnLTPUQlEZVmFBvWd5fv7vWTj4CTOs//sTlxQDvEnTS8X0/bqsdifGX3/FPfcC07qXNqxRUlERBSBFSUREVEEVpREREQRSs01yjWvtnWWl3Z41sbrD/nXK8vdUdPJp/MXFXldwdlIAKDZH5b55Qd+OwQnBgaAKm+7T4ahI1U4uZ6Nbz7l44Tes3pKc2e5DrakdZuCJk09z1keec2KkJxEZVPwuuRFH7i3QPWtus3GYzZ3sPG0tac5+XT28XHL7nvFp87yyCb+d0C/cdudtPxx/jXQC64aZuPgLSVA6bithC1KIiKiCKwoiYiIIpSarterW7ndmsEhx+sO+beAYHbRu1oBt7t15cPuw7qnNvQn+Q0+oHvdA6c4+bLBp/EUZtvPGtp4QLWpofmGfdfDxicHnoYEAIdQMk6r4j4gek6TXjY+tHptMW8NUWbsbu9fHhlWwz1Huy/6lY2Pu9A/L+thGRIx729u22th/W42Hj20kZN25gWLbfz+C/7EBRO2N3XyvXeNXwbmLEZJYIuSiIgoAitKIiKiCKWm6zXdyrd2u02X31jDxiv6TIjNbgXntKz++RonjTNQFm7L6VJ4JgCr7j/VxlV+KB2jiS+u6j5J6MFOdW1cjV2vxYrzT2ZO5Wn++XbxNPfh5sdhVWz2lBxav8HGDcducNK+H+vHHW690caxI2fvec2fTOHPvxnupFWYMS8NW1k4tiiJiIgisKIkIiKKwIqSiIgoQqm5RvnmmvbO8qgcfxhwh6w9Nu62aF9C5Z2R/Zaz3LOK/7782MwBNy8cYOP6m5YmtC7yHc4On0kgqLQ85Sg4gXRwRhMiKj4n/+1zGy98qYGTdtIHO2w87pmnnbTf/2WEjTM5GwlblERERBFYURIREUUoNV2vdQe7Q4f7vt3fxu+29J8eEeySLYpugeHH+QPd2wD+3f5lG9d+Ojup8slo23atjfMjO7lLh4Pq3/RzNGwvUVkXvKUEAKbcfr6NN451bxt6fPSjNv51g9/buOHYz5FObFESERFFYEVJREQUgRUlERFRhFJzjTJ/1y73hXP95V79b7Dx5o7hdXvN5f74/hovuX3ZW17Yb+MV7V910ibuyLVx9tKNNi6pWSyoZKw7dMBZrrLlQEhOIiouVab6t5ItnBd+68iC6x6xcd+xndO6DWxREhERRWBFSUREFKHUdL1Gyc7zJ0zOzUuujBW9nrFx7G0AE1aeY+N63yU2QSkdnYb2+zA07ZJnRznLDWemd4g5AVev627jyY0+Cc33zUNnOsucTYSAI28deXRhTxsPP2d1xtbLFiUREVEEVpREREQRjoqu12TETtwM+BN8xo5urPNo5WLYomPDnrvq2Xjus+WdtE5Z/lNwvp3SxsYNL0vuaUvJ6FzFnYx7zn5/ouncBxY6aXxOD1Epc0YbZ/GFMyfaeML2phlbLVuUREREEVhREhERRWBFSUREFKHMXqNcPaZSaNpl84c6y3Vnfp3pzTlmlPvXfBuPePh3TtpXt4638UddnrDxkJ43OfnKp/l4rHm1rY3PrjzPSTtr/kAb19rzn7Sul4yf+nex8eRGT5XgllCsdXef5SxX/p8f1xlfOm6PKt+qhY13jtvjpNWvsNfG7w/pFkhJ77gHtiiFr7+wAAADRUlEQVSJiIgisKIkIiKKUKa6XrVrOxu/0+XxmFT/FhCZXrOYtujYdtKsH53lTr0G23hu5xdtvL6He3tOo5mpr3vPpX533+td/Mldv9if5eSrdS9vDcq0xn9aXtKbQAFbf9PVxouHjnfSTp3lX5aq4yalrEKD+s7yuisbxs3X5CL3CTu3N3jFxrP3ureA9B/rP02r1ldfpLqJodiiJCIiisCKkoiIKAIrSiIioghl6hrl5s5Vbdy4gnvtKThjSIV9Csq8/EUrnOWT7/AfK5iXV8vG7wx5wMl3wQkjbdx8xJcIIx1b23hT1xpO2lM3+5O4nlrJ/z3YctowJ1+L2XNA6RW8HQRI/JaQbiN+a+NmeZwtpDhUFPcxk8t7+LMszV/jf2de+cV1Tj4JxN2bfGPjldtrO/lmtpli43Jwb/vKhwbS/BIf397YyTdwhv9/0WrsRiet1vrMXZcMYouSiIgoAitKIiKiCGWq63XfCX5TPnZy5od/bGXjnKeLp7lOrsNLV9r4+Qv8CVef+od7rN6/+EEbv96to41ffbmXk++ZYf749Q5Z4XN9XLBsgI1bPrHLSeMMIcWr6WvDbRw7GXM2wrvZKX1yJvrff2ftGe6kbe6zP+57nu860Vk+I8v/rg3O2pHvdMq6t5vkb3WfltYk72DcdVWa942z3GLnXBsfivuOzGOLkoiIKAIrSiIioghlqut1cL/wR7pMmnqejXPBrteSdmj1WhtnDTzRSRve4fc2rnjrDzaed+MjTr6W00aElt/4Lb9TNWvmIhvnHzwQLzulUXae24V6fl57GzcDR7OWJtVfnR2zHD/fOJyeYInupY2mmB+SL9zhwrMUO7YoiYiIIrCiJCIiisCKkoiIKEKZukb55hr/WsionPRO3EmZc3jLFme54oeB5Q/9sC86O/laILGn6vA5TESUCrYoiYiIIrCiJCIiilCmul51uv+g7dvruw9mrjO3NA46JiKi0o4tSiIiogisKImIiCKwoiQiIopQpq5R1nn0cxsvedRNq5LgrQRERERBbFESERFFYEVJREQUQVT53BIiIqIwbFESERFFYEVJREQUgRUlERFRBFaUREREEVhREhERRWBFSUREFOH/AZwWePTfrBGGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    ckpt = tf.train.get_checkpoint_state('./model_hw10')\n",
    "    if ckpt and ckpt.model_checkpoint_path:\n",
    "        saver.restore(sess, ckpt.model_checkpoint_path)\n",
    "        final_pred, acc = sess.run(\n",
    "            [pred, accuracy],\n",
    "            feed_dict={\n",
    "                x: mnist.test.images[:16],\n",
    "                y: mnist.test.labels[:16],\n",
    "                keep_prob: 1.0\n",
    "            })\n",
    "        orders = np.argsort(final_pred)\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        print(acc)\n",
    "        for idx in range(16):\n",
    "            order = orders[idx, :][-1]\n",
    "            prob = final_pred[idx, :][order]\n",
    "            plt.subplot(4, 4, idx + 1)\n",
    "            plt.axis('off')\n",
    "            plt.title('{}: [{}]-[{:.1f}%]'.format(\n",
    "                np.argmax(mnist.test.labels[idx]), order, prob * 100))\n",
    "            plt.imshow(mnist.test.images[idx].reshape((28, 28)))\n",
    "\n",
    "    else:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 模型构建及训练过程\n",
    "正确率稳定维持在 98% 以上。  \n",
    "该网络模型结构包括**两个卷积层，两个池化层和三个全连接层**，最后两层全连接层之间加入了 **dropout 提高模型的泛化能力**。   \n",
    "tensorflow 实现深度学习主要由两步构成。第一是模型的创建，通过构建计算图来实现，第二步是通过创建 session 会话训练模型。有了会话，就可以对变量 W， b 进行优化模型的训练，首先要通过正向传播计算当前输出与标注结果之间的损失函数，然后通过反向传播（选择合适的优化方法）调整每一层的参数。 \n",
    "在 softmax 分类模型中，通常使用“交叉熵（cross entropy）”损失来衡量这种相似性，损失越小模型的输出就和实际标签越接近，模型的预测也就越准确。\n",
    "在这个模型中，我们的损失函数为**交叉熵损失+7e-5\\*L2 损失**；优化方法是**梯度下降法**。  \n",
    "本模型的网络结构见 Tensorboard 可视化的计算图。 \n",
    "\n",
    "![Graphs](./logs_hw10/graphs.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积的特性\n",
    "  \n",
    "1. **局部相关性**：相邻像素结合起来表达一种特征，随着网络层数的加深，特征图中的点映射到原图中的感受野不断增大，表达的特征也由局部特征向全局特征变化。 \n",
    "2. **参数共享**：对于一张特征图上的任意一点，都是在原图上用相同的卷积核运算得到。 \n",
    "3. **平移宽容**：无论目标物在图像的哪一位置，都可以激活神经元。 \n",
    "4. **缩放宽容**：通常认为 25% 以内的缩放比例不会产生太大的干扰。 \n",
    "5. **少许降维**：当 padding=valid 时，或 stride > 1 时，卷积运算会降低特征维度。 \n",
    "6. **对输入尺寸不做要求**：卷积核固定的情况下，特征的 size 会随着输入尺寸的变化而变化。 \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 为什么卷积的效果要比全连接网络好，给出至少 2 点理由。\n",
    "1. **参数量**：\n",
    "    + 卷积共享参数，来提取特定方向上的特征，形成局部连接，每个神经元不再和上一层的所有神经元相连，而只和一小部分神经元相连，这样就减少了很多参数；全连接神经网络输入参数在像素矩阵是指数级增长。\n",
    "    + 权值共享，一组连接可以共享同一个权重，而不是每个连接有一个不同的权重，这样又减少了很多参数。\n",
    "    + 下采样，可以使用 pooling 来减少每层的样本数，进一步减少参数数量。\n",
    "2. **相关性**：图像的局部相关性较大，卷积操作可以更好地利用图像的局部相关性，而全连接层特征平铺为一维向量，切断了相邻像素点和特征点之间的关联，所以效果要差一些；换句话说，全连接神经网络无法很好地处理图像数据，然而卷积网络可以。\n",
    "3. **干扰抑制**：卷积操作无论目标在图像的哪个位置，都会被激活，对无效的背景信息可以起到抑制作用；而全连接网络雾大抑制图片中的干扰信息，影响预测的准确性。 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
